Evaluating Statistical Claims: Observational Studies and Experiments Flashcards

Researchers want to determine whether a new tutoring program improves standardized test scores. They randomly assign 150 students to receive tutoring and 150 students to a control group with no tutoring, then compare scores. Why is random assignment important in this study?

Answer: It controls for confounding variables by balancing them across groups on average.

Random assignment distributes known and unknown confounding variables roughly equally between groups, allowing differences in outcomes to be attributed to the treatment.

A study observes that neighborhoods with more parks have lower obesity rates. A city planner concludes that building more parks will reduce obesity. Which type of additional evidence would most strengthen the claim that parks reduce obesity?

Answer: A randomized experiment in which some neighborhoods are randomly assigned new parks

Only a randomized experiment with random assignment can establish causation; more observational evidence strengthens association but not causation.

A researcher surveys 400 college students randomly selected from one university about social media use and GPA. The results show a negative association. Which statement correctly describes what the researcher can conclude?

Answer: There is a negative association between social media use and GPA among students at this university.

Because this is an observational study with a sample from one university, the results can identify an association in that population but cannot establish causation or broad generalizability.

A clinical trial randomly assigns 500 patients to either a new drug or a placebo. The drug group shows significantly fewer symptoms. A doctor generalizes this to all patients with the condition. What concern is most valid?

Answer: The study can only be generalized to patients similar to those who participated in the trial.

Even with random assignment, generalizability is limited to populations represented by the study participants; the doctor's broad generalization may be unwarranted.

An education researcher randomly assigns 40 classrooms to use a new math curriculum and 40 to use the standard curriculum. At year-end, the new curriculum classrooms score significantly higher. What is the most valid conclusion?

Answer: The new curriculum causes higher math scores in classrooms like those in the study.

Random assignment of classrooms supports a causal conclusion for classrooms similar to those in the study, though it may not generalize to all possible classroom environments.

A company surveys all of its 200 employees and finds that those who participate in the wellness program have fewer sick days. The CEO announces that the wellness program reduces sick days. What is the most significant flaw in this conclusion?

Answer: Employees who are already healthier may be more likely to join the program, creating selection bias.

Without random assignment, healthier employees may self-select into the program; this selection bias means the program may not be causing the difference in sick days.

A study compares cancer rates in people who live near power lines versus those who do not, and finds a higher cancer rate near power lines. What is the most important limitation of this study?

Answer: The study is observational, so the higher cancer rate could be due to confounding variables rather than power line proximity.

Observational studies cannot rule out confounding variables (e.g., socioeconomic factors, industrial pollution) that may explain both proximity to power lines and higher cancer rates.