ASEP Verification and Validation 2 — Questions and Answers

Question 1: Which of the four standard verification methods involves operating the system and observing its behavior without formal measurement?

Test
Analysis
Inspection
Demonstration (Correct answer)

Correct answer: Demonstration

Demonstration involves operating the system to show it can perform a required function, using observation rather than formal measurement to confirm compliance.

Demonstration is one of four standard verification methods (Test, Analysis, Inspection, Demonstration—TAID). It involves operating the system under realistic conditions and observing that it performs the required function. Unlike testing, demonstration does not require formal measurement against quantitative thresholds—it relies on qualitative observation that the function is performed. Demonstration is appropriate for requirements that are qualitative in nature ('the system shall display a map') or where the threshold is obvious ('the system shall boot in under 60 seconds' can be demonstrated with a simple timer observation). It is less rigorous than testing but more efficient for requirements where measurement infrastructure would be disproportionate to the verification need.

Question 2: What is the purpose of a Verification Cross-Reference Matrix (VCRM)?

To list all project team members
To map each requirement to its planned verification method, level, event, and success criteria (Correct answer)
To track project budget
To document stakeholder contact information

Correct answer: To map each requirement to its planned verification method, level, event, and success criteria

A VCRM maps every requirement to its verification method, the level at which it will be verified, the verification event, and the success criteria for compliance.

A Verification Cross-Reference Matrix is a planning and tracking document that maps each system requirement to its planned verification approach. For each requirement, the VCRM specifies: the verification method (test, analysis, inspection, or demonstration), the verification level (component, subsystem, or system), the specific verification event or review where it will be verified, the success criteria for determining compliance, and the current verification status. The VCRM ensures completeness (every requirement has a planned verification), supports test planning by aggregating requirements by method and event, and provides a status dashboard for tracking verification progress throughout the program.

Question 3: Why is it important to plan verification methods during requirements definition rather than during the test phase?

It is not important—verification can be planned at any time
Early planning ensures requirements are written to be verifiable and that verification resources, facilities, and schedules are adequately planned (Correct answer)
It reduces the number of requirements
It simplifies the design process

Correct answer: Early planning ensures requirements are written to be verifiable and that verification resources, facilities, and schedules are adequately planned

Planning verification during requirements definition ensures requirements are verifiable as written and that adequate resources, facilities, and schedule are allocated for verification activities.

Planning verification methods during requirements definition serves two critical purposes. First, it ensures verifiability—if the planned verification method cannot confirm compliance with the requirement as written, the requirement must be rewritten before it is baselined. This prevents the costly situation of discovering during the test phase that requirements are ambiguous, unmeasurable, or untestable. Second, early verification planning enables adequate resource allocation: test facilities must be reserved, specialized equipment must be procured, analysis models must be developed, and test schedules must be integrated into the project plan. Deferring verification planning leads to compressed test schedules, inadequate test infrastructure, and incomplete verification—all significant program risks.

Question 4: What distinguishes system-level verification from component-level verification?

They are identical processes at different scales
System-level verification evaluates the complete integrated system against system requirements; component-level verification evaluates individual components against their allocated specifications (Correct answer)
Component verification is more important
System verification only checks documentation

Correct answer: System-level verification evaluates the complete integrated system against system requirements; component-level verification evaluates individual components against their allocated specifications

System-level verification evaluates the complete integrated system against system-level requirements, while component-level verification evaluates individual components against their specifications.

System-level and component-level verification address different levels of the requirements hierarchy. Component-level verification confirms that individual components meet their allocated specifications (derived from system requirements through decomposition and allocation). System-level verification confirms that the fully integrated system meets its system-level requirements, including emergent properties that only manifest when all components operate together. Some requirements can only be verified at the system level (end-to-end performance, system-level reliability, electromagnetic compatibility). The verification program must plan which requirements are verified at which level, ensuring complete coverage without unnecessary redundancy.

Question 5: What is a test procedure and what must it include to be effective?

A general description of the test approach
A step-by-step document specifying prerequisites, setup, execution steps, expected results, pass/fail criteria, and data recording requirements (Correct answer)
A list of test equipment only
A summary of requirements

Correct answer: A step-by-step document specifying prerequisites, setup, execution steps, expected results, pass/fail criteria, and data recording requirements

An effective test procedure provides step-by-step instructions with prerequisites, setup, execution steps, expected results, pass/fail criteria, and data recording requirements.

A test procedure is a detailed, step-by-step document that enables consistent, repeatable test execution. Effective test procedures include: prerequisites and preconditions (system state, prior tests completed, safety conditions), test setup instructions (equipment configuration, connections, software initialization), step-by-step execution instructions (specific actions in sequence with expected system responses), pass/fail criteria for each step and for the overall test, data recording requirements (what to measure, precision required, recording format), anomaly handling instructions, and safety warnings. Well-written procedures enable any qualified person to execute the test consistently, which is essential for repeatability, auditability, and independent verification.

Question 6: How does model-based verification complement traditional testing?

It replaces all physical testing
It enables verification of conditions difficult to test physically, supports early verification before hardware exists, and reduces the number of physical test cases needed (Correct answer)
It is less accurate than testing in all cases
It only works for software systems

Correct answer: It enables verification of conditions difficult to test physically, supports early verification before hardware exists, and reduces the number of physical test cases needed

Model-based verification complements testing by verifying conditions impractical to test physically, enabling early verification before hardware availability, and optimizing test programs.

Model-based verification uses mathematical models, simulations, and digital twins to complement physical testing. It offers several advantages: verification of extreme conditions that are dangerous, expensive, or impossible to create physically (e.g., nuclear detonation survivability, deep space radiation); early verification before hardware exists, enabling design corrections when they are cheapest; parametric analysis across wide ranges of operating conditions; verification of rare failure modes and edge cases; and optimization of physical test programs by identifying the most critical test cases. However, model-based verification requires validated models (models must be shown to accurately represent real system behavior), and it cannot fully replace physical testing for final system acceptance.

ASEP Verification and Validation 2

Which of the four standard verification methods involves operating the system and observing its behavior without formal measurement?

🃏 ASEP Flashcards — Study This Quiz as Flip Cards →

📚 Want to go deeper? Read the Full ASEP Study Guide →