Evaluating Statistical Claims: Observational Studies and Experiments Flashcards

Researchers conduct an observational study and find that adults who take multivitamins have lower rates of chronic illness. A supplement company advertises that multivitamins prevent chronic illness. What is the most critical flaw in this advertising claim?

Answer: Adults who take multivitamins may also engage in many other healthy behaviors, making it impossible to isolate the effect of vitamins alone.

People who take multivitamins are likely health-conscious in many ways, creating confounding between vitamin use and other health behaviors.

A study randomly assigns 200 patients to either a new drug or a placebo and finds that the drug group has significantly fewer hospital readmissions. A hospital administrator wants to use this result to project cost savings for all patients. What concern is most valid?

Answer: The study sample may not be representative of all patients, limiting the generalizability of cost savings estimates.

While random assignment supports the causal effect, projecting to all patients requires that the study sample represent the broader hospital population.

An observational study finds that people who attend religious services weekly have lower blood pressure than those who do not. A cardiologist claims that attending religious services reduces blood pressure. What is the most important limitation of this claim?

Answer: Frequent service attendees may have stronger social support networks and lower stress for reasons unrelated to religious attendance itself.

Religious service attendance is associated with social connection and community support, which independently lower stress and blood pressure, creating confounding.

A research group randomly assigns 50 elementary classrooms to use a new math game software and 50 to use traditional worksheets for one semester. Math test scores are significantly higher in the software group. What is the best causal conclusion?

Answer: Using the math game software caused higher math scores compared to worksheets in these classrooms.

Random assignment of classrooms to conditions supports the causal conclusion that the software (vs. worksheets) caused higher scores within this study.

A national survey finds that adults who cook at home more often have lower body mass index (BMI) than those who eat out frequently. A health campaign claims: 'Cook at home to lose weight.' What is the primary flaw in this campaign?

Answer: The survey is observational; people who cook at home may also make healthier food choices and exercise more, making it impossible to isolate cooking's effect.

Home cooking is correlated with other healthy behaviors; the lower BMI may be due to overall lifestyle differences rather than cooking location alone.

A study randomly assigns 200 adults to either an app-guided diet or an unguided diet for 12 weeks, with both groups consuming the same number of calories. The app group loses significantly more weight. Which statement is most accurate?

Answer: The app guidance caused more weight loss than unguided dieting at the same calorie level.

With both groups at the same calorie level and random assignment, the causal conclusion is that app guidance specifically caused the additional weight loss in this experiment.

Researchers observe that children who participate in after-school art programs have lower rates of disciplinary incidents at school. An educator concludes that art programs reduce disciplinary incidents. Which of the following most undermines this conclusion?

Answer: Students who join art programs may already be more engaged in school and less prone to disciplinary incidents for independent reasons.

Students who voluntarily join art programs may self-select based on already having better behavioral dispositions, creating a confound that explains the lower disciplinary rates.