ATP ATP Test Development & Design 2 — Questions and Answers
Question 1: Which standard-setting method involves panelists rating the difficulty of items for a 'borderline' examinee?
- Angoff Method (Correct answer)
- Bell Curve Method
- Z-Score Method
- Raw Score Method
Correct answer: Angoff Method
The Angoff method asks panelists to estimate the probability that a minimally competent examinee would answer each item correctly.
Question 2: What is the purpose of an 'item stem' in a multiple-choice question?
- To provide the answer key
- To present the question or problem to be solved (Correct answer)
- To list incorrect answer options
- To state the scoring rubric
Correct answer: To present the question or problem to be solved
The item stem is the part of a multiple-choice question that presents the problem or question the examinee must answer.
Question 3: What is 'differential item functioning' (DIF)?
- Items that are worth different point values
- A condition where items perform differently for subgroups with similar ability levels (Correct answer)
- Items that change content between test forms
- A scoring method for partial credit
Correct answer: A condition where items perform differently for subgroups with similar ability levels
DIF occurs when an item performs differently for subgroups (e.g., by gender or ethnicity) after controlling for overall ability.
Question 4: In test form construction, what is 'test equating'?
- Making all test forms identical
- A statistical process to ensure scores from different forms are comparable (Correct answer)
- Ensuring equal numbers of items per subject
- Administering the same test to equal numbers of candidates
Correct answer: A statistical process to ensure scores from different forms are comparable
Test equating is a statistical process that adjusts scores from different test forms so they can be compared on a common scale.
Question 5: What is the role of a 'subject matter expert' (SME) in test development?
- To program the testing software
- To provide domain knowledge for item writing and review (Correct answer)
- To administer the test to examinees
- To handle test registration logistics
Correct answer: To provide domain knowledge for item writing and review
Subject matter experts provide domain knowledge and professional judgment during item writing, review, and standard setting.
Question 6: What is an 'anchor test' in equating designs?
- A test used exclusively for high-stakes decisions
- A common set of items administered across multiple test forms to link them statistically (Correct answer)
- The oldest version of a test in a bank
- A backup test used when primary forms are unavailable
Correct answer: A common set of items administered across multiple test forms to link them statistically
An anchor test contains common items shared across forms and serves as the statistical link for equating different test versions.
ATP ATP Test Development & Design 2
Which standard-setting method involves panelists rating the difficulty of items for a 'borderline' examinee?