MINUTES OF THE

SIGIST STANDARDS WORKING PARTY MEETING

HELD ON 23^rd April 200 2

1. Present:              Brian Hambling, Graham Thomas, John Kent, Margaret Saner, Martin Nixon, Simon Tansey, Steve Allott, Stuart Reid.

2. Apologies:        Colin Upton, David Ramsey, Isabel Evans, Laurel Saunders, Marco Giromini, Marq Harding, Trevor Price.

3. Agree Agenda:

02/4.1                      The agenda was agreed.

4. Minutes from the Previous Meeting:

02/4.2                      The minutes from the previous meeting were agreed.

5. Actions from the Previous Meetings:

00/6.5                      Guidelines Section

(01/3.5)                   Stuart has agreed to amend the guidelines section of the draft Standard to ensure that it includes a section on how testers cope with poor or non-existent requirements.
Carried Forward

SR

01/1.10                    Teams

                                A number of people have joined/left the teams since the last meeting.  The teams as of 12^th April 2002 are:

01/10.4                   Circulate Glossary of Terms for Review

                                The Performance Testing Team has produced a glossary of terms relating to their technique. The meeting proposed that all teams produce a glossary of terms as part of the standard. This will be collated by e-mail, posted on the web site for review, with an additional meeting arranged to review the contents.
[22/02/02 Graham Thomas has contacted all Team Leaders and they have given a schedule for the production of a glossary of terms for their technique plus their availability for a glossary of terms review meeting. The consensus for a meeting was not before June 2002.]
[23/04/02 Graham has arranged a Glossary Review meeting for 25^th June 2002.]
Carried Forward.

All Team Leaders

01/11.6                    Circulate Umbrella Standards Document
It was stated that references to other techniques will be included as a global section of the standard. Stuart will circulate the umbrella standards document to the group.
This action has been revised to post the Umbrella Standards Document on the website (www.tesitngstandards.co.uk), in the procedures section.
[27/02/02 John Kent will e-mail Stuart Reid requesting the umbrella document.]
Carried Forward

JK

01/12.5                   Interoperability Testing - Update the Interoperability Testing Technique
The team agreed to

·         to check that the OED definition of communication did not require it to be two way

·         to document how to detail attributes of the interoperability interface

·         to document how to work out appropriate techniques for test case generation

Carried Forward.

PS/HF

02/1.4                      Performance Testing

                                A number of amendments were discussed and Julian agreed to organise the necessary changes.

                                Closed.

JH

02/1.5                      Procedure Testing

·         Reference the IEEE standard on Software Reviews (IEEE 1028-1997).

·         Include Model Office as a technique.

                                Closed.

BH

02/1.6                      Test Patterns

                                Julian agreed to write a proposal for a BCS SIGIST sub group to investigate test patterns and forward this to Stuart for presentation at the next BCS SIGIST committee meeting.

                                Carried Forward.

JH

02/2.3                      Compatibility Testing - Meeting Actions
- Investigate the use of State Transition, Boundary Conditions and configuration audits.
- Investigate the set of compatibility tasks; shared components, events & triggers, operating system time slicing, cause & effect and sequencing of activities e.g. installation.
- Include cross references to the interoperability and memory management techniques.
Carried Forward.

PS

02/2.4                      Conversion Testing – Definition of Scope
Gareth agreed to put together a definition of the scope of Conversion Testing and circulate for review.
Closed.

GP

02/2.05                    Team Leaders – Reaffirm commitment to team
The team leaders present suggested that all team leaders should write out to their team members to re-affirm the commitment required from the team.
Closed.

Team Leaders

02/2.6                      Management Meeting
Graham suggested that the Standards Working Party should hold a management meeting to determine the next steps and plan out work remaining to complete the non-functional testing techniques.
Carried Forward.

GT

02/2.7                      New Joiners Guide
Isabel Evans suggested the production of a guide for new joiners which could be posted on the Web Site. As Secretary Graham has direct contact with all new members so will look at this.
Carried Forward.

GT

02/3.3                      Portability Testing - Meeting Actions
- Include an example which covers the porting of code from one environment to another.
- Include in the guidelines the fact that people use the term compatibility to describe the portability testing example documented.
- Include investigative analysis and exploratory testing in the Portability Testing guidelines.
- Update the technique to reflect the comments received in the meeting.
New Action.

JK

02/3.4                      Recovery/Disaster Recovery Testing – Meeting Actions
- Define the term Business Continuity in the glossary section.
- Include techniques in the design section for DR.
- Include inspection as a technique.
- Include in the technique that DR testing is testing the DR plan and not testing that we have recovered after a disaster.
New Action.

5. Techniques Review

5.1 Security Testing

                                Brian Hambling briefly recapped the work of the team over the last three months, including the changes to the team members during the period.
He then outlined the changes that had been made to the individual techniques, and the team members then walked the meeting through the changes.

                                The following actions were recorded for the Security Testing Team:

02/4.3                      Security Testing - Meeting Actions
- Review BS7799 for a list of vulnerabilities.
- Investigate the use of the Syntax Testing technique.
- Identify the difference between Configuration Auditing and Security Configuration Auditing.
- Secure State Transition needs to be developed, with investigation into whether this is a formal technique for inclusion.
- Obtain a copy of the Edvice document from Julian Harty (Performance/Stress Team).
- Apply comments as noted in the meeting.
New Action.

BH

5.2 Performance/Stress Testing

                                Steve Allott presented an update of the progress that the Performance/Stress Team had made since their last meeting. They have now split their work into 4 distinct documents; Normative, Guidelines, Glossary and Reference Material.
Analysis and execution has been excluded form the techniques. They have also described a ‘Performance Test Case’ and the difference between white and black box performance testing.

                                Julian Harty then walked the meeting through the following techniques; Hardware Limits, Queue and Buffer Analysis (QBA), Data Driven & Query design, Industry Benchmarks (IB), and Traversal Interactivity.

                                The following actions were recorded for the Performance/Stress Testing Team:

02/4.4                      Performance/Stress Testing – Meeting Actions
- Remove the Data Driven and Query Design technique from the standard.
- Clarify the wording and the use of the term ‘accuracy’.
- Investigate the Operational Profiling technique.
- Apply edits to the normative as detailed in the meeting
New Action.

SA/JH

6. Any Other Business:

02/4.5                      Management Meeting
Request agenda items from the Team Leaders for the Management Meeting to be held in June.

GT

02/4.6                      Further BS Testing Standards

                                Stuart Reid volunteered to apply to the BCS SIGIST Committee to sponsor a seat on the BSI committee to further the BS Testing Standards.

SR

                                Graham thanked PA Consulting for their continued support in the provision of meeting rooms.

                                Graham also thanked all of the volunteers who responded to the request for assistance with the Recovery/Disaster Recovery and Security Teams.

02/4.7                      Website Updates

                                Graham and John Kent agreed to agreed to set aside some time to discuss pending updates to the Standards Working Party website www.testingstandards.co.uk.

GT/JK

7. Next Meeting:

02/4.8                     The updated list of meetings is shown below and is also available on the SWP website. The next meeting is a management meeting to plan the work remaining for the group to complete the Non-Functional Testing Techniques standard. All are welcome to attend, although priority for places at the meeting will go to the committee and Team Leaders.

                                All scheduled Team meetings will take place at PA (Consulting), 123 Buckingham Palace Road, London, between 14:00 and 17:00.