MINUTES OF THE

SIGIST STANDARDS WORKING PARTY MEETING

HELD ON 15^th January 2002

1. Present:              Brian Hambling, Dave Norman, Graham Thomas, John Kent, Julian Harty, Martin Nixon, Stuart Reid.

2. Apologies:        Angela Edwards, David Ramsey, Geoff Bailey, Isabel Evans, Kath Harrison, Steve Allott.

3. Agree Agenda:

02/1.1                      Julian Harty asked if the agenda could be amended to include a discussion on test patterns. The meeting agreed and a discussion on test patterns was added to the agenda to follow the techniques section.

4. Minutes from the Previous Meeting:

02/1.2                      The minutes from the previous meeting were agreed.

5. Actions from the Previous Meetings:

00/6.5                      Guidelines Section

(01/3.5)                   Stuart has agreed to amend the guidelines section of the draft Standard to ensure that it includes a section on how testers cope with poor or non-existent requirements.
Carried Forward

SR

01/5.4                      Recovery/DR Testing –Examples

(01/5.7, 01/5.8)       The team will simplify the examples, include automated recovery, and rewrite the Normative and Guidelines sections.
Carried Forward.

JA

01/5.9                      SWP Administration

(01/5.10)                 John agreed to investigate putting links on the website to other testing sites and will contact them by e-mail.
Closed.

02/1.3                      SWP Administration

                                John agreed to separate the website links to other testing sites into non-profit making organisations and commercial organisations.  He also agreed to add a disclaimer stating that the working party were not implicitly recommending particular organisations by including links to their websites on our website.
New Action.

JK

01/6.4                      Procedure Testing – TMAP
Check TMAP to see if there is anything relevant to standards work.
Closed.

01/1.10                    Teams

                                A number of people have joined/left the teams since the last meeting.  The teams as of 21^st January 2002 are:

01/6.6                      Procedure Testing – Actions
- Reference will be made to the fact that functionality is being implicitly tested when the Procedure is being tested,
- Pursue Steve Allott for reference details on Inspection and Review techniques,
- Investigate references on Auditing techniques, and End-to-End process testing,
- Measure the number of Procedures not the number of Procedure steps (consider Procedures which have multiple exit points),
- Make greater use of analogies, and be consistent with the use of “Procedure” and “Process”, in our Procedure Testing material.
- We should reference the Process Cycle Technique (ex. Martin Pol).
Closed.

01/7.3                      Interoperability Testing - Draft Standard
The team is to generate a draft normative and guidelines section based upon the minuted diagrams.
Carried Forward.

RH-Y

01/10.4                   Circulate Glossary of Terms for Review

                                The Performance Testing Team has produced a glossary of terms relating to their technique. The meeting proposed that all teams produce a glossary of terms as part of the standard. This will be collated by e-mail, posted on the web site for review, with an additional meeting arranged to review the contents.
[15/01/02 Graham Thomas will follow up the Team Leaders by e-mail].
Carried Forward.

All Team Leaders

01/10.5                   Performance Testing - Update Performance Testing Examples

                                The meeting suggested that the Performance Testing Team use the examples contained in the reviewed draft document as the basis for worked examples of how to generate test cases.
Closed.

01/11.6                    Circulate Umbrella Standards Document
It was stated that references to other techniques will be included as a global section of the standard. Stuart will circulate the umbrella standards document to the group.
This action has been revised to post the Umbrella Standards Document on the website (www.tesitngstandards.co.uk), in the procedures section.
Carried Forward

JK

01/12.3                   Portability Testing - Update the Portability Testing Technique
The team agreed to update the technique;

·         to detail how the test cases are derived

·         to document how portability issue lists are generated

·         to change 'platforms' to 'environments'

·         to define 'environments'

·         to include portability of interpreted languages as well as compiled

·         to remove example 2

·         to define the term 'system behaviours'

·         to expand example 1

Carried Forward.

                              JK/JW/FM/AS

01/12.4                   Portability Testing - Produce diagrams and document the definitions
John Kent agreed to produce a document with completed definitions and diagrams for Portability, Interoperability Configuration and Compatibility as discussed in the meeting.

                JK

01/12.5                   Interoperability Testing - Update the Interoperability Testing Technique
The team agreed to

·         to check that the OED definition of communication did not require it to be two way

·         to document how to detail attributes of the interoperability interface

·         to document how to work out appropriate techniques for test case generation

PS/HF

5. Techniques Review

5.1 Performance Testing

                                Julian Harty presented the revisions that the Performance team had made since the last meeting to the performance material. These included the first 5 pages of the documentation, including the normative clause. There was a general discussion about what material should be placed in the additional examples section, to be published on the website. Then the meeting reviewed the Performance Normative clause in detail. 

02/1.4                      Performance Testing

                                A number of amendments were discussed and Julian agreed to organise the necessary changes.

                                                                                                                                                                                                JH

                                Julian also presented the Performance Triangle, cost, performance & hardware, to aid in understanding the constraints when carrying out performance testing.

5.2 Procedure Testing

                                Graham gave a brief progress report since the last procedure testing meeting. This included feedback from the workshop at Eurostar 2001.

                                Brian Hambling has joined the SWP and offered assistance with producing the standards. He will take over from Graham Thomas as Team Leader of team 4 Security and Procedure Testing, leaving Graham free to focus on his role as secretary of the SWP.

                                The meeting then reviewed the draft Procedure Testing normative and the following actions were recorded:

02/1.5                      Procedure Testing

·         Reference the IEEE standard on Software Reviews (IEEE 1028-1997).

·         Include Model Office as a technique.

BH

5.3 Test Patterns

                                Julian gave a brief introduction to the performance testing patterns document that he had circulated prior to this meeting. Stuart agreed that test patterns were of interest to the testing world. He suggested that because test patterns were very new and therefore not yet suitable for incorporation in testing standards that the best way to investigate them may be through a BCS SIGIST sub group. Stuart agreed that if Julian produced a proposal for a test patterns sub group, he would present it at the next BCS SIGIST committee meeting on February 11th.

02/1.6                      Test Patterns

                                Julian agreed to write a proposal for a BCS SIGIST sub group to investigate test patterns and forward this to Stuart for presentation at the next BCS SIGIST committee meeting.

JH

6. Any Other Business:

                                Geoff Bailey wrote to the SWP about the standards production process. This subject will be added as an agenda item for the next meeting.

                                There are a number of failed e-mail addresses in the SWP distribution list. Graham requested that if any member of the SWP has an up to date e-mail address for any of the following members, that they forward it to him:

·         Richard Durham

·         Gerry Schmidt

·         Maurice Blackman

·         David Smiles

·         Graham Freeburn

·         Stephen Cooper

                                Isabel Evans had noted that the next SWP meeting clashes with the BCS SIGIST meeting and had asked whether it was possible to reschedule. Graham will investigate.

7. Next Meeting:

01/11.9                   The updated list of meetings is shown below and is also available on the SWP website. The next meeting involves teams 5 and 6, who will be considering Compatibility Testing and Conversion Testing.

                                NB. The date of the next meeting has been changed to Friday 25^th of February.

                                All scheduled Team meetings will take place at PA (Consulting), 123 Buckingham Palace Road, London, between 14:00 and 17:00.