Reliability testing shall use a model of the software/system that specifies its required level of reliability (e.g. mean time to failure, rate of occurrence of failure). The model shall include a definition of failure and either the operational profile of the software/system or the means to derive it.
Test cases shall be designed to satisfy the operational profile of the software/system and shall be run on the complete software/system. Depending on the form of the reliability requirement, either the ‘time to failure’ or the ‘number of transactions to failure’, or similar shall be recorded. When the software/system fails the fault that led to the failure shall be documented and corrected. Testing shall continue until sufficient measures are available to predict that the required level of reliability has been achieved (see Measurement).
The software/system’s reliability shall be measured using the record of the testing (see Design) and a suitable reliability growth model. The rationale for the choice of reliability growth model shall be documented.