ATP Item Development & Banking 2 — Questions and Answers
Question 1: What is differential item functioning (DIF) analysis primarily used to detect?
- Items that are too difficult for the overall examinee population
- Items that perform differently for subgroups with equal ability levels (Correct answer)
- Items that have low correlation with total test scores
- Items that exceed the recommended word count
Correct answer: Items that perform differently for subgroups with equal ability levels
DIF analysis identifies items where examinees of equal ability but different demographic subgroups (e.g., gender, ethnicity) have systematically different probabilities of answering correctly, flagging potential bias.
Question 2: In item response theory (IRT), what does the 'b' parameter represent?
- Item discrimination
- Item difficulty or location on the ability scale (Correct answer)
- Pseudo-guessing probability
- Item information value
Correct answer: Item difficulty or location on the ability scale
In IRT, the b parameter (difficulty or location parameter) indicates the point on the ability scale where an examinee has a 50% probability of answering the item correctly.
Question 3: Which item bank management practice helps maintain test security when many test forms are needed over time?
- Reusing the same items across all test administrations
- Tracking item exposure rates and retiring overexposed items (Correct answer)
- Publishing item statistics publicly after each administration
- Allowing examinees to take notes during testing
Correct answer: Tracking item exposure rates and retiring overexposed items
Monitoring and controlling item exposure rates prevents frequently seen items from becoming known to future examinees, protecting the integrity and validity of the test.
Question 4: What is the purpose of anchor items (common items) when equating scores across test forms?
- To serve as bonus items that raise examinee scores
- To provide a set of identical items across forms that link the forms to a common scale (Correct answer)
- To replace field test items that performed poorly
- To measure examinees' test-taking strategies rather than content knowledge
Correct answer: To provide a set of identical items across forms that link the forms to a common scale
Anchor items appear on multiple test forms and allow scores to be placed on a common scale, compensating for differences in form difficulty during equating.
Question 5: Which index is commonly used in classical test theory to assess how well an item separates higher-ability examinees from lower-ability examinees?
- Point-biserial correlation (item-total correlation) (Correct answer)
- Cronbach's alpha
- Standard error of measurement
- Item difficulty (p-value)
Correct answer: Point-biserial correlation (item-total correlation)
The point-biserial correlation measures the relationship between performance on a single item and total test score, indicating how well the item discriminates between high and low performers.
Question 6: When reviewing items for sensitivity concerns, what is the primary goal?
- To ensure items are sufficiently challenging for graduate-level examinees
- To identify and remove content that may offend, stereotype, or disadvantage particular groups (Correct answer)
- To verify that items include culturally specific references to increase engagement
- To check that items use complex vocabulary appropriate for the subject matter
Correct answer: To identify and remove content that may offend, stereotype, or disadvantage particular groups
Sensitivity reviews identify language, images, or scenarios that could be offensive, stereotyping, or culturally biased in ways that unfairly disadvantage certain groups of examinees.
Question 7: What is a 'key' in the context of multiple-choice item development?
- A numerical code assigned to an item in the item bank
- The correct answer to a multiple-choice item (Correct answer)
- The rubric used to score constructed-response items
- The thematic category to which an item belongs
Correct answer: The correct answer to a multiple-choice item
In testing terminology, the 'key' refers specifically to the correct answer option among the choices provided in a multiple-choice item.
ATP Item Development & Banking 2
What is differential item functioning (DIF) analysis primarily used to detect?