Test Validation and Reliability Flashcards

Which type of validity evidence examines whether test scores correlate with an external criterion measured at the same time?

Answer: Concurrent validity

Concurrent validity is established when test scores correlate with a criterion measure collected simultaneously.

A reliability coefficient of 0.50 for a selection test would generally be considered:

Answer: Too low for most selection purposes

A reliability coefficient of 0.50 is generally considered too low for employment selection tests, which typically require 0.70 or higher.

What does the standard error of measurement (SEM) primarily tell us?

Answer: The expected variation in an individual's scores across repeated administrations

SEM estimates how much an individual's observed score is likely to vary if the test were administered multiple times.

When a test is used to predict future job performance measured six months after hiring, this is an example of:

Answer: Predictive validity

Predictive validity involves collecting criterion data after a time delay following test administration.

Which reliability method requires only a single test administration?

Answer: Internal consistency reliability

Internal consistency methods (e.g., Cronbach's alpha, KR-20) assess reliability using data from a single test administration.

Differential item functioning (DIF) analysis is used to identify items that:

Answer: Perform differently for subgroups after matching on ability

DIF flags items that favor or disadvantage a particular subgroup after controlling for overall ability level.

In classical test theory, a person's observed score is defined as:

Answer: True score plus measurement error

Classical test theory states that Observed Score = True Score + Error Score.