Evaluating Statistical Claims: Observational Studies and Experiments Flashcards

An observational study finds that people who own houseplants have lower cortisol levels (a stress hormone) than those who do not. A wellness company claims: 'Buying houseplants will lower your stress.' What is the key flaw in this claim?

Answer: The observational study cannot prove causation; houseplant owners may have lower-stress lifestyles for other reasons.

An observational study cannot rule out the possibility that people who tend to own houseplants also engage in other stress-reducing behaviors.

Researchers randomly assign 80 hospital patients to listen to calming music during recovery and 80 to recover in silence, then measure recovery time. Patients in the music group recover 15% faster. Which conclusion is best supported?

Answer: Listening to calming music during recovery caused faster healing in this controlled experiment.

Random assignment in a controlled experiment supports the conclusion that music caused faster recovery in this study population.

A survey of 5,000 adults finds that those who volunteer have higher self-reported happiness than non-volunteers. A community leader concludes that volunteering makes people happier. Which design improvement would provide the strongest evidence for this causal claim?

Answer: Randomly assign participants to volunteer or not volunteer and compare happiness after 6 months.

Only a randomized experiment—where participants are randomly assigned to volunteer or not—can control for self-selection and establish causation.

A drug company conducts a randomized trial in which 600 patients are randomly assigned to receive a new pain reliever or a placebo. After 4 weeks, the drug group reports significantly less pain. A physician generalizes this result to all patients with chronic pain. What is the most important concern with this generalization?

Answer: The trial participants may not be representative of all chronic pain patients.

Even with random assignment (which supports causation), random selection of participants from the target population is needed to generalize results broadly.

A social scientist observes that countries with higher chocolate consumption have more Nobel Prize winners per capita. The scientist writes: 'Eating chocolate may boost cognitive function leading to Nobel Prizes.' What is the primary flaw in this reasoning?

Answer: The ecological correlation may reflect confounding factors (e.g., wealth) rather than a causal effect of chocolate.

Country-level correlations often reflect underlying confounders like national wealth, education spending, and research infrastructure—not a direct effect of chocolate consumption.

An educational researcher randomly assigns 50 schools to adopt a new bilingual curriculum and 50 schools to continue with their existing curriculum for one year. Schools with the new curriculum show significantly higher English proficiency scores. What is the most appropriate conclusion?

Answer: The new bilingual curriculum caused higher English proficiency scores in this experiment.

Random assignment of schools to curriculum conditions supports a causal claim about the curriculum's effect within the study context.

A longitudinal observational study follows 8,000 adults for 25 years and finds that those who regularly engage in hobbies live longer. A newspaper headline reads: 'Hobbies add years to your life.' Which of the following is the most serious concern about this headline?

Answer: The headline implies causation, but the observational design cannot rule out that healthier people are more likely to have hobbies.

People who are healthier may both live longer and have more capacity to engage in hobbies, making the direction of causation unclear in this observational study.