NCMHCE Research and Program Evaluation 1 — Questions and Answers
Question 1: A counselor wants to determine whether a new CBT-based group therapy program is reducing depression symptoms compared to a waitlist control. Which research design is MOST appropriate?
- Case study design
- Randomized controlled trial (RCT) (Correct answer)
- Phenomenological study
- Grounded theory study
Correct answer: Randomized controlled trial (RCT)
An RCT with random assignment to treatment and control groups provides the strongest evidence for causal inference about program effectiveness.
Question 2: Which of the following best describes the concept of internal validity in counseling research?
- The degree to which findings can be generalized to other populations
- The extent to which an instrument measures what it claims to measure
- The confidence that observed changes are due to the independent variable rather than extraneous factors (Correct answer)
- The consistency of results across repeated administrations of a measure
Correct answer: The confidence that observed changes are due to the independent variable rather than extraneous factors
Internal validity refers to the degree to which the researcher can conclude that the independent variable (e.g., treatment) caused the observed changes, ruling out alternative explanations.
Question 3: A clinician reviews research showing that a therapy is effective across multiple randomized trials. Which level of evidence does this represent according to the evidence-based practice hierarchy?
- Expert opinion and clinical experience
- Single case experimental design
- Systematic review or meta-analysis of RCTs (Correct answer)
- Descriptive cohort study
Correct answer: Systematic review or meta-analysis of RCTs
Systematic reviews and meta-analyses synthesizing multiple RCTs represent the highest level of evidence in the evidence-based practice hierarchy.
Question 4: When a researcher uses a p-value of .05 as the threshold for statistical significance, what is being controlled?
- The probability of a Type II error
- The probability of a Type I error (Correct answer)
- The statistical power of the study
- The effect size of the intervention
Correct answer: The probability of a Type I error
A p-value threshold of .05 controls the Type I error rate — the probability of incorrectly rejecting a true null hypothesis (false positive).
Question 5: A researcher finds a statistically significant result (p < .01) but notes a very small effect size (d = .10). What is the most accurate interpretation?
- The treatment effect is both statistically and clinically meaningful
- Statistical significance alone confirms practical importance of the findings
- The result is likely a Type II error due to low power
- The effect may be statistically significant but of limited practical or clinical significance (Correct answer)
Correct answer: The effect may be statistically significant but of limited practical or clinical significance
Statistical significance does not equal clinical meaningfulness; a small effect size indicates the observed difference has minimal practical impact even if statistically detectable with a large sample.
Question 6: Which qualitative research approach involves exploring the lived experience of a phenomenon from the participant's perspective?
- Grounded theory
- Phenomenology (Correct answer)
- Ethnography
- Case study
Correct answer: Phenomenology
Phenomenology is a qualitative method focused on understanding the essence of a lived experience as described by those who have had it.
Question 7: A counseling program director wants to assess whether clients are achieving their individualized treatment goals. Which evaluation method is MOST directly aligned with this purpose?
- Needs assessment
- Goal attainment scaling (GAS) (Correct answer)
- Cost-benefit analysis
- Fidelity assessment
Correct answer: Goal attainment scaling (GAS)
Goal Attainment Scaling (GAS) is specifically designed to measure client progress toward individually defined treatment goals, making it ideal for this purpose.
NCMHCE Research and Program Evaluation 1
A counselor wants to determine whether a new CBT-based group therapy program is reducing depression symptoms compared to a waitlist control.
Which research design is MOST appropriate?