HSRT Evaluating Scientific Evidence 1 — Questions and Answers
Question 1: A researcher claims that a new drug reduces blood pressure based solely on a study of 12 patients over two weeks. Which limitation most significantly weakens this evidence?
- The drug has not been approved by the FDA
- The sample size is too small to generalize findings (Correct answer)
- The study was not published in a peer-reviewed journal
- The patients were not required to change their diet
Correct answer: The sample size is too small to generalize findings
A sample of 12 patients is far too small to draw reliable generalizations, making the findings ungeneralizable to broader populations.
Question 2: Which study design provides the strongest evidence for establishing a causal relationship between an intervention and a health outcome?
- Cross-sectional survey
- Case report
- Randomized controlled trial (Correct answer)
- Cohort study
Correct answer: Randomized controlled trial
Randomized controlled trials minimize confounding through random assignment, making them the gold standard for establishing causality.
Question 3: A meta-analysis of 20 studies concludes that vitamin D supplementation reduces fracture risk. One study in the analysis had 10,000 participants while the others averaged 200. What is the most appropriate concern?
- The meta-analysis should not include RCTs
- The large study may disproportionately influence the pooled result (Correct answer)
- Meta-analyses cannot address fracture outcomes
- The smaller studies should be excluded entirely
Correct answer: The large study may disproportionately influence the pooled result
In meta-analyses, a single large study can dominate the pooled estimate, potentially skewing results if that study has unique methodological features.
Question 4: A study reports a statistically significant result with p = 0.04. What does this indicate?
- There is a 4% probability the null hypothesis is true
- The result has a 4% chance of occurring if the null hypothesis is true (Correct answer)
- The effect size is clinically meaningful
- The study will replicate with 96% certainty
Correct answer: The result has a 4% chance of occurring if the null hypothesis is true
A p-value of 0.04 means there is a 4% probability of observing a result this extreme or more extreme assuming the null hypothesis is true.
Question 5: Researchers find a strong positive correlation (r = 0.85) between ice cream sales and drowning rates. The most accurate interpretation is:
- Eating ice cream increases drowning risk
- A third variable (hot weather) likely explains both (Correct answer)
- Drowning causes people to crave ice cream
- The correlation proves a direct causal link
Correct answer: A third variable (hot weather) likely explains both
Correlation does not imply causation; a confounding variable like hot weather explains both increased ice cream sales and increased swimming activity.
Question 6: Which of the following best describes 'publication bias' in the context of medical research?
- Journals preferentially publish studies with positive or significant results (Correct answer)
- Researchers selectively report only favorable patient outcomes
- Funding agencies refuse to support negative studies
- Peer reviewers reject studies from unknown institutions
Correct answer: Journals preferentially publish studies with positive or significant results
Publication bias occurs when journals disproportionately publish studies showing positive or statistically significant results, skewing the literature.
Question 7: An observational study concludes that coffee drinkers have lower rates of type 2 diabetes. A reviewer notes that coffee drinkers in the study also exercised more. This is an example of:
- Recall bias
- Selection bias
- Confounding (Correct answer)
- Information bias
Correct answer: Confounding
Confounding occurs when a third variable (exercise) is associated with both the exposure (coffee) and the outcome (diabetes), distorting the apparent relationship.
HSRT Evaluating Scientific Evidence 1
A researcher claims that a new drug reduces blood pressure based solely on a study of 12 patients over two weeks.
Which limitation most significantly weakens this evidence?