MINUTES OF THE

SIGIST STANDARDS WORKING PARTY MEETING

HELD ON 21^st May 2002

1. Present:              Graham Thomas, Isabel Evans, John Kent, Steve Allott, Stuart Reid.

2. Apologies:        Angela Edwards, Brian Hambling, David Ramsey, Jane Allerton, Phil Skinner, Mark Harding.

3. Agree Agenda:

02/5.1                      The agenda was agreed, with the addition of a point to discuss the Umbrella Standards Document as point 7.

4. Minutes from the Previous Meeting:

02/5.2                      The minutes from the previous meeting were agreed.

5. Actions from the Previous Meetings:

00/6.5                      Guidelines Section

(01/3.5)                   Stuart has agreed to amend the guidelines section of the draft Standard to ensure that it includes a section on how testers cope with poor or non-existent requirements.
Carried Forward

SR

01/1.10                    Teams

                                A number of people have joined/left the teams since the last meeting.  The teams as of 12^th April 2002 are:

** To be confirmed

01/10.4                   Circulate Glossary of Terms for Review

                                The Performance Testing Team has produced a glossary of terms relating to their technique. The meeting proposed that all teams produce a glossary of terms as part of the standard. This will be collated by e-mail, posted on the web site for review, with an additional meeting arranged to review the contents.
[22/02/02 Graham Thomas has contacted all Team Leaders and they have given a schedule for the production of a glossary of terms for their technique plus their availability for a glossary of terms review meeting. The consensus for a meeting was not before June 2002.]
[23/04/02 Graham has arranged a Glossary Review meeting for 25^th June 2002.]
[12/06/02 The glossary review meeting has now been rescheduled for 16^th July 02]
Carried Forward.

All Team Leaders

01/11.6                    Circulate Umbrella Standards Document
It was stated that references to other techniques will be included as a global section of the standard. Stuart will circulate the umbrella standards document to the group.
This action has been revised to post the Umbrella Standards Document on the website (www.tesitngstandards.co.uk), in the procedures section.
[27/02/02 John Kent will e-mail Stuart Reid requesting the umbrella document.]
Carried Forward

JK

01/12.5                   Interoperability Testing - Update the Interoperability Testing Technique
The team agreed to

·         to check that the OED definition of communication did not require it to be two way

·         to document how to detail attributes of the interoperability interface

·         to document how to work out appropriate techniques for test case generation

Carried Forward.

PS/HF

02/1.6                      Test Patterns

                                Julian agreed to write a proposal for a BCS SIGIST sub group to investigate test patterns and forward this to Stuart for presentation at the next BCS SIGIST committee meeting.

                                Closed.

02/2.3                      Compatibility Testing - Meeting Actions
- Investigate the use of State Transition, Boundary Conditions and configuration audits.
- Investigate the set of compatibility tasks; shared components, events & triggers, operating system time slicing, cause & effect and sequencing of activities e.g. installation.
- Include cross references to the interoperability and memory management techniques.
Carried Forward.

PS

02/2.6                      Management Meeting
Graham suggested that the Standards Working Party should hold a management meeting to determine the next steps and plan out work remaining to complete the non-functional testing techniques.
Closed.

02/2.7                      New Joiners Guide
Isabel Evans suggested the production of a guide for new joiners which could be posted on the Web Site. As Secretary Graham has direct contact with all new members so will look at this.
Carried Forward.

GT

02/3.3                      Portability Testing - Meeting Actions
- Include an example which covers the porting of code from one environment to another.
- Include in the guidelines the fact that people use the term compatibility to describe the portability testing example documented.
- Include investigative analysis and exploratory testing in the Portability Testing guidelines.
- Update the technique to reflect the comments received in the meeting.
Carried Forward.

JK

02/3.4                      Recovery/Disaster Recovery Testing – Meeting Actions
- Define the term Business Continuity in the glossary section.
- Include techniques in the design section for DR.
- Include inspection as a technique.
- Include in the technique that DR testing is testing the DR plan and not testing that we have recovered after a disaster.
Closed.

02/4.3                      Security Testing - Meeting Actions
- Review BS7799 for a list of vulnerabilities.
- Investigate the use of the Syntax Testing technique.
- Identify the difference between Configuration Auditing and Security Configuration Auditing.
- Secure State Transition needs to be developed, with investigation into whether this is a formal technique for inclusion.
- Obtain a copy of the Edvice document from Julian Harty (Performance/Stress Team).
- Apply comments as noted in the meeting.
Carried Forward.

BH

02/4.4                      Performance/Stress Testing – Meeting Actions
- Remove the Data Driven and Query Design technique from the standard.
- Clarify the wording and the use of the term ‘accuracy’.
- Investigate the Operational Profiling technique.
- Apply edits to the normative as detailed in the meeting
Carried Forward.

SA/JH

02/4.5                      Management Meeting
Request agenda items from the Team Leaders for the Management Meeting to be held in June.
Closed

02/4.6                      Further BS Testing Standards

                                Stuart Reid volunteered to apply to the BCS SIGIST Committee to sponsor a seat on the BSI committee to further the BS Testing Standards.
Stuart has applied to the committee to sponsor this position and this has been accepted.
Closed

02/4.7                      Website Updates

                                Graham and John Kent agreed to agreed to set aside some time to discuss pending updates to the Standards Working Party website www.testingstandards.co.uk.
Closed

5. Progress to date and remaining workload (Team by Team):

5.1 Team 1 Configuration/Portability

                                John Kent reported progress and discussed the way that the team had been working over the previous months. This led the meeting on to discussing the ways of working, and the agenda continued at Item 7.

                                The remaining progress reports are extracted from comments during the meeting and e-mail reports.

5.1 Team 1 Recovery/Disaster Recovery

                                A comprehensive written report was received complete including updates to the Recovery and Disaster Recovery techniques plus a glossary section.

5.1 Team 1 Performance/Stress

                                Steve Allott reported that the team was in the middle of preparing a revised draft of the Performance Stress documentation which was due for release before the next SWP meeting.

5.1 Team 1 Security/Procedure

                                Submission prior to meeting by e-mail.

5.1 Team 1 Interoperability/Compatibility

                                Submission prior to meeting by e-mail

5.1 Team 1 Installability/Conversion

                                Isabel Evans reported that the Installability technique was complete and the draft should be put up on the website in the completed techniques section for comment.

02/5.3                      Update Website – Completed Technique
Graham was actioned to forward the latest copy of the Installability technique for John Kent to post on the Website as a completed technique for review and comment.
New Action.

GT/JK

5.2 Revisions to completed techniques

                                There was not time to discuss this topic.

6. New Techniques:

6.1 Endurance Testing

                                There was not time to discuss this topic.

7. Ways of working:

                                The meeting discussed at length the way that the two technique team structure had been operating. The discussion was wide ranging, covering the workloads, preparation for and attending meetings, presentations in the meetings, commitment of team members, production of deliverables, reviewing output, and the overall deliverable that we are working towards.

                                The consensus view of the meeting was that in terms of producing the techniques for the standard, it is actually easier to produce domain examples first and then produce the normative. And that a lot of effort had been expended producing normative sections which then had to be rewritten to accommodate findings from producing the examples.

                                This was an important point, and a way into making it easier to produce the standard, and reduce the amount of repetitive rewriting of parts of the documentation this speeding up the process.

                                The following revisions to the ways of working were proposed:

                                Overview

                                For each technique the outstanding work will be split into domains. An initial list of domains is below, and will be brainstormed at the next SWP meeting on 25/06/02;

                                - Safety Critical
- Banking/Financial
- Command & Control
- Office
- Process Control
- E-business.

                                Starting in September the group will hold one meeting a month, focussed on a specific domain, to review examples for each technique for that domain. Each team will be asked to send a representative to the meeting to present their technique examples.

                                The interim deliverables for this stage will be hyperlinked web based documents, delivered in word which can be converted for presentation on the website.

                                Extra meetings will be arranged after the domain review meetings to cover; glossary review, risk and the umbrella document.

                                And a meeting will be arranged to review all outstanding draft normatives.

                                Detail

                                To support this way of working each team is asked to complete the following planning form, which identifies their outstanding actions, the contributors, the due date and the deliverables. The teams will also be asked to maintain these plans and provide regular progress reports.

                                As a precursor to the planning activity the next SWP meeting to be held on the 25^th June 02, initially scheduled for a glossary review meeting will become part two of the management process, where the team leaders can identify the matrix relating their techniques to domains.

                                Actions

                                In order to implement the revised approach the following actions were identified:

02/5.4                      SWP Planning Actions
- Re-arrange glossary review meeting for 16^th July 02.
- Reschedule 25^th June 02 meeting as a planning meeting.
- Change start of 25^th June 02 to 3:00pm.
- Request Team Leaders or deputy to attend planning meeting.
- Include brainstorming of application domains on agenda for planning meeting.
New Action.

GT

02/5.5                      SWP General Actions

8. Agree future meeting schedule:

                                The meeting agreed to defer the agreement of the future meeting schedule until the next SWP meeting.

9Any other business:

02/5.6                      Glossary Update
Stuart requested that the term Test Condition be added to the glossary.
New Action.

GT

02/5.7                      SWP Reference Material
Stuart Reid will find out the cost of gaining access to the BSI and IEEE websites for access to published standards for the SWP research purposes.
New Action.

SR

02/5.8                      References to BS7925-1/2
Stuart reported that people were still e-mailing him direct requesting copies of the standards.
Graham agreed to investigate.
[Post Meeting Note 12/06/02 The BCS SIGSIST Website still points people looking for the standard direct to Stuart. The BCS SIGIST webmaster has been requested to amend these references to point to the www.testingstandards.co.uk website.]
New Action.

GT

10. Date of Next Meeting:

The updated list of meetings is shown below and is also available on the SWP website. The next meeting is a continuation of the management meeting to plan the work remaining for the group to complete the Non-Functional Testing Techniques standard. All are welcome to attend, although priority for places at the meeting will go to the committee and Team Leaders.

                                All scheduled Team meetings will take place at PA (Consulting), 123 Buckingham Palace Road, London, between 14:00 and 17:00.