Adaptive Testing and Computer Adaptive Testing (CAT) Flashcards

Which of the following is a major criticism of using Maximum Information (MI) item selection without additional constraints in CAT?

Answer: It tends to overexpose a small subset of highly informative items

Unconstrained MI selection repeatedly chooses the same high-information items for examinees at similar ability levels, depleting those items quickly and creating security vulnerabilities.

Compared to a traditional paper-and-pencil fixed-form test, a well-designed CAT of equivalent precision typically requires:

Answer: Fewer items to achieve the same reliability

Because CAT targets each item to the examinee's ability level, it achieves equivalent measurement precision with roughly 50% fewer items compared to a fixed-form test.

Polytomous IRT models (e.g., Graded Response Model) are used in CAT when:

Answer: Items have multiple ordered score categories (partial credit)

Polytomous models like the GRM or Partial Credit Model handle items that can be scored in more than two ordered categories, such as constructed-response or rating-scale items.

Which regulatory or professional body publishes standards that address fairness and validity concerns specific to adaptive testing?

Answer: The Standards for Educational and Psychological Testing (AERA/APA/NCME)

The Standards for Educational and Psychological Testing, jointly published by AERA, APA, and NCME, provides authoritative guidance on validity, fairness, and reliability requirements applicable to adaptive tests.

In a CAT, construct-irrelevant variance introduced by adaptive item selection can occur when:

Answer: The adaptive algorithm systematically exposes certain subgroups to different content domains

If subgroup differences in ability lead the algorithm to route groups to systematically different content areas, test content may inadvertently vary across groups, threatening construct validity and fairness.

The conditional standard error of measurement (CSEM) in IRT-based CAT is best described as:

Answer: The precision of the ability estimate at a specific theta value

Unlike CTT's single SEM, IRT-based CSEM varies along the ability continuum and indicates how precisely theta is estimated at each specific ability level.

Which of the following best supports the argument that equating is less of a concern in CAT than in fixed-form testing?

Answer: IRT ability estimates are placed on a common scale during item calibration, making form-to-form equating largely unnecessary

Because all CAT items are calibrated onto a common IRT theta scale, scores are inherently comparable across administrations without the additional equating step required to link different fixed forms.