MINUTES OF THE

SIGIST STANDARDS WORKING PARTY MEETING

HELD ON 16^th SEPTEMBER 2002

1. Present:              Graham Thomas, Isabel Evans, John Kent, Mark Harding, Steve Allott, Stuart Reid.

  2. Apologies:        David Ramsey, Jane Allerton, Jerry Stubbs, Julian Harty, Lloyd Roden, Margaret Saner.

3. Agree Agenda:

 02/9.1                      The agenda was agreed.

4. Minutes from the Previous Meeting:

 02/9.2                      The minutes from the previous meeting were agreed.

5. Actions from the Previous Meetings:

 01/1.10                    Teams

                                A number of people have joined/left the teams since the last meeting.  The teams as of 11^th September 2002 are:

(It is noted that Team 5 is currently without a Team Leader)

NB. Existing team actions will be retained in the minutes as an aide memoir for the teams during the production of the Domain Technique examples. These actions can be closed upon request.

01/12.5                   Interoperability /Compatibility Testing – Technique Updates
The team agreed the following Interoperability actions:

·         to check that the OED definition of communication did not require it to be two way

·         to document how to detail attributes of the interoperability interface

·         to document how to work out appropriate techniques for test case generation

The team agreed to the following Compatibility actions:

·         Investigate the use of State Transition, Boundary Conditions and configuration audits.

·         Investigate the set of compatibility tasks; shared components, events & triggers, operating system time slicing, cause & effect and sequencing of activities e.g. installation.

·         Include cross references to the interoperability and memory management techniques.

[The actions from 02/2.3 have been combined into this action point]
Carried Forward.

02/2.7                      New Joiners Guide
Isabel Evans suggested the production of a guide for new joiners which could be posted on the Web Site. As Secretary Graham has direct contact with all new members so will look at this.
[30/07/02] The new joiners guide will be modified to fit in with the new approach.
Carried Forward.

GT

02/3.3                      Portability Testing - Meeting Actions
- Include an example which covers the porting of code from one environment to another.
- Include in the guidelines the fact that people use the term compatibility to describe the portability testing example documented.
- Include investigative analysis and exploratory testing in the Portability Testing guidelines.
- Update the technique to reflect the comments received in the meeting.
Carried Forward.

JK

02/4.3                      Security Testing - Meeting Actions
- Review BS7799 for a list of vulnerabilities.
- Investigate the use of the Syntax Testing technique.
- Identify the difference between Configuration Auditing and Security Configuration Auditing.
- Secure State Transition needs to be developed, with investigation into whether this is a formal technique for inclusion.
- Obtain a copy of the Edvice document from Julian Harty (Performance/Stress Team).
- Apply comments as noted in the meeting.
Carried Forward.

BH

02/4.4                      Performance/Stress Testing – Meeting Actions
- Remove the Data Driven and Query Design technique from the standard.
- Clarify the wording and the use of the term ‘accuracy’.
- Investigate the Operational Profiling technique.
- Apply edits to the normative as detailed in the meeting
Carried Forward.

SA/JH

02/5.5                      SWP General Actions

 02/5.6                      Glossary Update
Stuart requested that the term Test Condition be added to the glossary.
Carried Forward.

GT

02/6.3                      Installability/Compatibility Testing Team Leader
Steve Allott agreed to provide contact details for volunteers to lead the Interoperability/Compatibility Testing team.
Carried Forward.

SA

02/7.3                      Post Umbrella Document on Website
Isabel will update the skeleton umbrella document and forward to John for posting to the website.
[Document produced. Superseded by action 02/9.4]
Closed.

02/7.4                      Review effectiveness of process for identifying glossary entries
Graham will table an agenda item for a future meeting to review the effectiveness of the process for identifying and reviewing glossary entries.
Carried Forward.

GT

02/7.5                      Include existing (Completed) techniques in domain technique matrix
Graham was asked to include the existing (completed) techniques for Memory Management, Reliability, Usability and Maintainability in the domain technique matrix.
Closed.

02/7.6                      Team leaders review domain technique matrix
Graham agreed to circulate the domain technique matrix to the team leaders so that they could determine which examples already existed, were not applicable, or were required.
Closed.

02/7.7                      Circulate domain technique matrix to working party
Graham agreed to circulate the domain technique matrix to the members of the working party asking each member to identify the domain technique examples for which they had experience/expertise of. This would include a cover note explaining the new approach.
[16/09/02 – Graham agreed to circulate the domain technique matrix to the working party again because of the e-mail problems that he had experienced (see 5.1.).]
Carried Forward.

GT/Working Party

02/7.8                      Generate domain technique example process flow
Isabel volunteered to generate a process flow for the production of domain technique examples.
Closed.

5. Domain Technique/Matrix Update

5.1 Required Techniques

                                Graham explained to the meeting that he had just found out that he was experiencing problems with his e-mail provider when sending e-mails out to the Standards Working Party which meant that not everyone had received the domain/technique matrix e-mails.

                                Graham gave a brief explanation of the domain/technique matrix that had been e-mailed out to members of the working party for them to highlight their areas of expertise/experience, and also mailed to the team leaders for them to identify which domain/technique examples were required.

                                Graham explained that in producing the matrix he had omitted to remove the command & control domain; however there weren’t any responses in this domain, which has now been removed from the matrix. The matrix now consists of 11 domains, 15 techniques and 165 domain/technique examples

                                The following table summarises the feedback from the matrix for required techniques:

·         165 Total Number of Domain/Technique examples

·         15 Already Exist

·         60 Required

·         24 Not Applicable

·         66 Unknown (blank entries)

                                There were some inconsistencies with the responses, where several techniques identified as not-applicable by the team leaders were identified as areas where members of the group had specific experience/expertise in testing.

5.2 Volunteer experience/expertise

                                Graham also gave a brief overview of the e-mail sent to all members of the Standards Working Party requesting them to fill in the matrix highlighting the domain/technique’s that they had specific expertise/experience of working in which could be used as a basis for generating examples.

                                The following table summarises the feedback form the individual members of the working party:

·         165 Total Number of Domain/Technique examples

·         68 covered by one or more members of the group

·         97 blank.

                                The key areas that the group was lacking knowledge in were the Safety Critical, Process Control, Scientific, Graphics and Knowledge Based Systems domains.

                                The discussion also ranged around how to increase the groups experience/expertise level in the above domains, with several members making suggestions as to groups to contact for potential recruits.

                                The following actions were recorded:

02/9.3                      Domain/Technique Matrix

6. Plan Approach to Domain Technique Examples

6.1 Umbrella Document

                                Isabel had produced a draft version of the Umbrella Document (the document which binds all the techniques together) to be posted on the website for review by all

02/9.4                      Website Admin
John agreed to post the Umbrella Document to the Standards Working Party website.
New Action.

JK

6.2 Glossary

                                The Glossary was not discussed at this meeting.

6.3 Generating Examples

                                The meeting spend time discussing the best way to use the domain/technique matrix responses in identifying who could generate examples.

                                The meeting agreed the following key points for generating domain/technique examples:

·         We would ask one person to produce an example at a time.

·         A three month timescale for production of examples seemed reasonable.

·         Examples should be about 2 sides of A4 in length.

·         Examples should be understandable by a tester who isn’t familiar with that area, but does have testing knowledge.

·         Examples may include other domains, but not other techniques.

·         The examples should be sanitised so that the complexity is simplified to convey the meaning, as long as this is not detrimental to the application of the technique.

·         Volunteers would be asked to provide timescales for the production of examples, and whether or not they needed assistance.

·         The production of examples would be monitored on a regular basis.

                                The meeting then spent time reviewing the responses to the matrix and drafting an initial list of candidates who would be asked if they would volunteer to produce the first set of examples.

02/9.5                      Generating Examples
Graham agreed to e-mail the candidates asking if they would be willing to volunteer to write a domain/technique example and within what timescale.
New Action.

GT

6.4 Review Meeting Schedule

                                The meeting felt that until a number of domain/technique examples had been produced and we understood how long it was taking, that it would be difficult to arrange a future review meeting schedule. It was agreed to defer this item for 3 months.

6.5 Deliverables

                                Steve suggested that the group should publicise the new approach and volunteered to produce a marketing plan.

02/9.6                      Marketing Plan
Steve agreed to produce a marketing plan for the Standards Working Party to publicise the new approach that the group was taking.
New Action.

SA

6.6 Website Updates

                                The website is seen as a key part of the new approach. John volunteered to put together a plan for the way the website could be used to enable group working and presentation of examples for review. Isabel agreed to assist with the process and ways of working for the group.

02/9.7                      Website Updates
John agreed to put together a plan to upgrade the website from its current form to include the domain/technique matrix and enable group working.
New Action.

JK

7. Any other business:

                                Stuart suggested that he could set-up an e-mail redirection to ensure that e-mails reached all members of the Standards Working Party.

02/9.8                      E-mail Redirection
Stuart agreed to set-up e-mail redirection for the Standards Working Party.
New Action.

SR

8. Date of Next Meeting:

The date for the next SWP meeting is listed below. At this meeting the 2003 meeting schedule will be determined. An updated list of meeting dates is also available on the SWP website

NB. All are welcome to attend.

                                All scheduled Team meetings will take place at PA (Consulting), 4^th Floor, 123 Buckingham Palace Road, London.