Evaluating Statistical Claims: Observational Studies and Experiments Flashcards

A study randomly assigns 150 restaurant workers to wear name badges and 150 not to wear them, then measures customer tip amounts over 4 weeks. Workers with badges receive significantly higher tips. What is the most valid conclusion?

Answer: Wearing a name badge caused an increase in tip amounts in this randomized experiment.

Random assignment controls for worker characteristics, supporting the conclusion that wearing a name badge caused the increase in tips within this experiment.

Researchers observe that students who take private music lessons perform better academically than those who do not. A school board member argues that offering free music lessons to all students will boost academic performance. Which limitation is most important?

Answer: Students who take private music lessons may come from families with greater financial resources and higher educational expectations.

Family income and educational expectations are confounders that may explain both music lesson participation and academic performance in this observational study.

A pharmaceutical company tests a new drug using a randomized double-blind placebo-controlled trial with 1,000 patients. The drug group shows significantly fewer symptoms. What does the double-blind design add to the study?

Answer: It prevents both patients and researchers from knowing who received the drug, reducing expectation-based biases.

A double-blind design means neither participants nor outcome assessors know who received the drug, eliminating placebo effects and observer bias.

A wellness researcher finds that adults in rural areas have lower rates of depression than adults in urban areas. A health official concludes that moving people to rural areas would reduce depression. What is the primary concern with this conclusion?

Answer: Rural and urban residents may differ in many ways (income, social networks, occupation) that affect depression rates beyond where they live.

This is an observational ecological comparison; rural and urban residents differ in many ways, making it impossible to attribute depression rate differences solely to living location.

Researchers randomly assign 80 college students to a daily aerobic exercise group and 80 to a stretching-only group for 10 weeks, then compare GPA. The aerobic group has a significantly higher GPA. Which conclusion is most valid?

Answer: Aerobic exercise caused a GPA improvement compared to stretching in this experimental sample.

Random assignment supports the causal claim that aerobic exercise (relative to stretching) caused the GPA improvement in this experiment.

An observational study finds that patients who frequently visit their doctor have lower mortality rates. A health insurer concludes that requiring more frequent doctor visits will reduce mortality. What design change would best test this causal claim?

Answer: Randomly assign patients to required frequent visits vs. standard care and compare mortality over 10 years.

Random assignment of visit frequency would control for the fact that healthier, more health-conscious patients may both visit doctors more often and have lower mortality.

An experiment recruits adult volunteers online and randomly assigns them to read either positive or neutral news articles for one month, then measures their self-reported optimism. The positive news group reports significantly higher optimism. What is the most important limitation?

Answer: Volunteer recruitment from an online platform may produce a self-selected sample that differs from the general population.

Even with random assignment (which supports causation), online volunteer recruitment may yield participants who are already more responsive to media interventions than the general population, limiting generalizability.