Adaptive Testing and Computer Adaptive Testing (CAT) Flashcards

The ability estimate (theta) in a CAT is updated after each item response using:

Answer: Maximum Likelihood Estimation (MLE) or Bayesian estimation

CAT systems iteratively update theta using MLE or Bayesian methods (such as EAP or MAP) after each item response to refine the ability estimate.

Shadow testing in CAT refers to:

Answer: Constructing a complete ghost test that satisfies all constraints before selecting the next item

Shadow testing builds a full, constraint-satisfying test in the background at each step and selects the next item from that shadow test, ensuring all blueprint and exposure constraints will be met.

A CAT that uses a three-parameter logistic (3PL) IRT model accounts for which additional parameter compared to the 1PL (Rasch) model?

Answer: Item discrimination and guessing (pseudo-chance)

The 3PL model adds discrimination (a) and pseudo-guessing (c) parameters to the difficulty (b) parameter of the Rasch model, accounting for the probability of correct response by low-ability examinees through guessing.

Early termination rules in mastery-based CAT (e.g., licensure tests) are triggered when:

Answer: The algorithm is sufficiently confident the examinee is above or below the passing standard

Mastery CATs use sequential probability ratio tests or similar decision rules to stop early when the probability of misclassifying the examinee falls below an acceptable error rate.

Which of the following best describes item parameter drift in the context of CAT?

Answer: A change in an item's statistical properties over time due to repeated exposure or population shifts

Item parameter drift occurs when an item's IRT parameters (b, a, c) change from their calibrated values, often because the item has been exposed and memorized or because the examinee population has shifted.

A CAT item bank is considered adequate when it:

Answer: Has sufficient items across the ability continuum with controlled exposure rates to support reliable adaptive administration

An adequate CAT item bank must have enough items at varying difficulty levels, across all content domains, and in sufficient quantity so that exposure limits can be enforced without depleting available items.

In CAT, the concept of 'theta initialization' refers to:

Answer: Choosing the starting ability estimate for the first item selection

Theta initialization sets the assumed starting ability level (often at the mean or a neutral value) used to select the first item before any examinee response data is available.