General Test Criteria
The scope of the test specification is unit test, integration test, and qualification test. These specifications should include the following criteria.
G.1: definition of test environment
G.2: definition of test procedure
G.3: definition of test case
Note: The selection of test cases shall have sufficient coverage according to e.g. the software integration test strategy and the release plan.
G.4: definition of verification criteria
Note: Verification criteria define the qualitative and quantitative criteria for verification of a requirement.
Unit Test Criteria
Create software unit test specification, and test the software units according to the unit test specification.
The unit test specification shall be suitable to provide evidence for compliance of the software units with the software detailed design and with the non-functional requirements.
U.1: Unit test specification should include the following criteria.
U.1.1: unit test cases
U.1.2: unit test data
U.1.3: static verification
U.1.4: coverage goals
U.1.5: coding standards (e.g. MISRA rules)
U.2: Verify that software unit conforms to the software detailed design.
U.2.1: input data format
U.2.2: output data format
U.2.3: prototype
U.2.4: flow chart
U.2.5: sequence chart
U.2.6: entity relationship diagram
U.2.7: pseudo code, etc.
Integration Test Criteria
Create software integration test specification, and test the integrated software according to the integration test specification.
The software integration test specification shall be suitable to provide evidence for compliance of the integrated software items with the software architectural design.
Note: Compliance to the architectural design means that the specified integration tests are suitable to prove that the interfaces between the software units and between the software items fulfill the specification given by the software architectural design.
I.1: Verify that inter-task/inter-process communication conforms to software architectural design.
I.2: Verify that the relationship and dependency between software elements conform to software architectural design.
I.3: Verify that the interrupts with their priorities conform to software architectural design.
I.4: Verify that the tasks with cycle time and priority conform to software architectural design.
I.5: Verify that the relationship between software elements conforms to software architectural design.
I.6: Verify that where the data (such as parameters) are stored and which measures (e.g. checksums, redundancy) are taken to prevent data corruption conform to software architectural design.
I.7: Verify that which data is persistent and under which conditions conforms to software architectural design.
Qualification
Develop specification for software qualification test. The test specification shall be suitable to provide evidence for compliance of the integrated software with the software requirements.
Q.1: Verify that the dynamic behavior of the software conforms to requirement specification. The dynamic behavior of the software means Start-up, shutdown, software update, error handling and recovery, etc.
Q.2: Verify that the relationship between the system elements and the software conforms to requirement specification.
Q.3: Verify that how variants for different model series or configurations are derived conforms to requirement specification.
(分かりづらいのでdependancyとかの表現に変えたい…)
Q.4: Verify that integrated software conforms to requirement specification.
Q.5: Verify that integrated software conforms to reliability requirements.
Q.6: Verify that integrated software conforms to security requirements.
Q.7: Verify that integrated software conforms to diagnosis requirements.
Q.8: Verify that integrated software conforms to software requirements specification.
Note4: Especially, Verify that integrated software conforms points below.
-any required software performance characteristics
-any required software interfaces
-any required security characteristics required
-any database design requirements
-any required error handling and recovery attributes
Q.9: On test environment, verify that integrated software conforms to resource consumption characteristics.