Research & Evidence-Based Practice Flashcards
7 cards from real ATP practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.
Read the first 7 Research & Evidence-Based Practice flashcards as text
Which approach to validity argument, articulated by Messick, emphasizes that validity is a unitary concept focused on:
Answer: The appropriateness of inferences and actions based on test scores
Messick's unified conception holds that validity is a single integrated judgment about the degree to which evidence supports the intended interpretations and uses of test scores.
A test developer discovers that a high-stakes certification exam has a significant adverse impact ratio below 4/5 for one ethnic group. The NEXT recommended step is to:
Answer: Investigate whether the score differences reflect construct-relevant or irrelevant factors
The 4/5 rule triggers further investigation, not automatic action; the critical question is whether group differences stem from real construct-related factors or test bias.
Which reliability approach is most appropriate for assessing inter-rater agreement on a constructed-response item with ordinal scoring rubrics?
Answer: Weighted kappa
Weighted kappa accounts for ordinal scoring by penalizing disagreements proportionally to their distance, making it more appropriate than simple kappa or Pearson r for rubric-scored responses.
Generalizability theory extends classical test theory by partitioning score variance into multiple facets, such as:
Answer: Person, item, rater, and occasion effects
G-theory designs allow simultaneous estimation of variance components attributable to persons, items, raters, occasions, and their interactions.
A licensing board wants to set a defensible passing score. Which of the following best strengthens the legal defensibility of the standard-setting process?
Answer: Documenting the qualifications of panelists and the procedural steps followed
Legal defensibility depends on transparent documentation of panelist qualifications, training, procedures, and rationale, demonstrating due diligence in the standard-setting process.
Which ethical obligation do test publishers have when research reveals that an operational test measure may be functioning as intended for some populations but not others?
Answer: Disclose findings, investigate causes, and take corrective action
Professional and ethical standards require publishers to disclose limitations, investigate differential functioning, and implement corrective measures to protect test-taker welfare.
Which research synthesis method allows quantitative aggregation of effect sizes across multiple independent validity studies?
Answer: Meta-analysis
Meta-analysis statistically combines effect sizes (e.g., validity coefficients) from multiple studies, providing a more precise and generalizable estimate than any single study.