Evaluating Statistical Claims: Observational Studies and Experiments Flashcards

A study randomly assigns 300 adults to participate in a strength training program or a stretching-only program. After 12 weeks, the strength training group shows greater bone density. The researcher concludes strength training causes increased bone density. Which additional question is most important for evaluating this conclusion?

Answer: Are the study participants representative of the broader adult population of interest?

While the randomized design supports causation within the study, you must consider whether the participants represent the target population to assess whether results can be generalized.

A poll of 1,200 randomly selected U.S. voters finds that 54% favor a specific immigration policy. Which conclusion is best supported?

Answer: A majority of U.S. voters likely favor this policy, subject to a margin of error.

A random sample supports a generalization about the broader voting population, but the result is an estimate with uncertainty (margin of error), not a precise count or a prediction of outcomes.

A researcher notices that ice cream sales and drowning rates are highly correlated throughout the year. She concludes that ice cream causes drowning. What is the most likely explanation for this correlation?

Answer: A confounding variable — hot weather — increases both ice cream sales and swimming (and therefore drowning).

Hot weather increases both ice cream consumption and swimming frequency, creating a spurious correlation between ice cream and drowning that has no causal mechanism.

A randomized clinical trial shows that Drug X reduces migraines better than a placebo in a diverse national sample of 2,000 adults. A neurologist concludes that Drug X should be the first-line treatment for all adults with migraines. Which statement about this conclusion is correct?

Answer: The conclusion is reasonable but should be further evaluated against side effects, cost, and results in subgroups.

Even strong randomized trial evidence supports — but does not fully determine — treatment decisions; real-world factors like side effects, cost, and subgroup differences must also be evaluated.

A high school conducts an experiment where 10 volunteering teachers adopt a new grading method and their students are compared to students of non-participating teachers. Students in the new grading method classes earn higher grades. What is the most significant flaw?

Answer: Teachers who volunteered may be more skilled or motivated, confounding the relationship between grading method and grades.

Volunteer teachers may be more effective or motivated regardless of the grading method, making teacher quality a confounding variable in this observational design.

Based on an observational study, a health organization claims that people who eat more vegetables have fewer depressive episodes. Which of the following, if true, would most weaken a causal interpretation?

Answer: People with lower depression are more likely to have energy and motivation to prepare and eat healthy meals.

If lower depression leads to better dietary habits (reverse causation), then depression may be causing the vegetable intake difference rather than the other way around.

A study randomly selects 400 schools nationwide and observes that schools with larger libraries have higher reading proficiency. A policy advisor concludes that expanding libraries will improve reading proficiency. What is the best evaluation of this recommendation?

Answer: The recommendation is premature because the observational study cannot establish that larger libraries cause higher proficiency.

Even with a nationally representative random sample, an observational study can only show association; confounding variables (e.g., school funding, community investment) may explain both library size and reading proficiency.