NCMHCE Research and Program Evaluation 5 — Questions and Answers
Question 1: A researcher asks participants to rate their own mental health on a 1-10 scale before and after counseling. What is the primary measurement concern with this approach?
- Lack of construct validity
- Social desirability bias inflating self-reported improvement (Correct answer)
- Insufficient statistical power
- Violation of random assignment
Correct answer: Social desirability bias inflating self-reported improvement
Self-report scales are particularly vulnerable to social desirability bias, where participants rate themselves more positively to appear well-adjusted.
Question 2: In qualitative research, which technique involves returning transcripts or summaries to participants so they can verify accuracy?
- Triangulation
- Thick description
- Member checking (Correct answer)
- Theoretical saturation
Correct answer: Member checking
Member checking (respondent validation) involves sharing findings with participants to confirm the researcher's interpretations are accurate.
Question 3: A program evaluator wants to understand the day-to-day operations of a crisis hotline to identify implementation barriers. Which evaluation approach is most fitting?
- Outcome evaluation using standardized scales
- Process evaluation examining program activities and fidelity (Correct answer)
- Cost-benefit analysis comparing program expenditures to outcomes
- Meta-analysis of published hotline studies
Correct answer: Process evaluation examining program activities and fidelity
Process evaluation examines how a program is implemented, its activities, and fidelity to the original design, identifying operational barriers.
Question 4: Which of the following is the correct sequence in the research process?
- Data collection → Literature review → Research question → Data analysis
- Research question → Literature review → Methodology → Data collection → Analysis (Correct answer)
- Methodology → Research question → Data collection → Literature review
- Literature review → Data analysis → Research question → Methodology
Correct answer: Research question → Literature review → Methodology → Data collection → Analysis
The standard research sequence begins with a question, situates it in existing literature, selects a methodology, collects data, and then analyzes results.
Question 5: A counseling agency uses program outcome data showing reduced client symptom scores to justify continued funding to a state board. This is an example of which evaluation purpose?
- Formative evaluation for program improvement
- Accountability evaluation for stakeholder reporting (Correct answer)
- Exploratory research for hypothesis generation
- Developmental evaluation for adaptive management
Correct answer: Accountability evaluation for stakeholder reporting
Accountability evaluation uses outcome data to demonstrate program effectiveness to funders, policymakers, or oversight bodies.
Question 6: A researcher studying trauma uses grounded theory methodology. Data collection stops when new interviews no longer produce new themes. This is called:
- Purposive saturation
- Theoretical saturation (Correct answer)
- Data triangulation
- Confirmability
Correct answer: Theoretical saturation
Theoretical saturation in grounded theory is the point at which additional data no longer generate new categories or modify existing theory.
Question 7: When interpreting a correlation coefficient of r = -0.72 between counselor burnout scores and client satisfaction ratings, a counselor-researcher should conclude:
- Burnout causes lower client satisfaction
- There is a strong negative relationship, but causation cannot be inferred (Correct answer)
- The relationship is weak and clinically insignificant
- The negative sign indicates a data entry error
Correct answer: There is a strong negative relationship, but causation cannot be inferred
A correlation of -0.72 reflects a strong negative association, but correlational designs cannot establish causation due to potential confounders.
NCMHCE Research and Program Evaluation 5
A researcher asks participants to rate their own mental health on a 1-10 scale before and after counseling.
What is the primary measurement concern with this approach?