Evaluating Statistical Claims: Observational Studies and Experiments Flashcards

A study finds that employees who use ergonomic chairs report fewer back problems than those who use standard chairs. Participants chose which chair to use. A company concludes that providing ergonomic chairs will reduce employee back problems. What is the primary concern with this conclusion?

Answer: Employees who chose ergonomic chairs may have been more health-conscious and engaged in other practices that reduce back problems.

Self-selection into chair type creates a confound; employees who seek out ergonomic chairs may also exercise more or take other preventive actions.

Researchers randomly assign 250 participants to a mindfulness program and 250 to a control group (waiting list) for 8 weeks, then measure anxiety levels. The mindfulness group shows significantly lower anxiety. To which group can the results best be generalized?

Answer: People similar to those who enrolled in the study

Generalizability is determined by how the sample was selected; the results apply most directly to people similar to those who enrolled, not to all people with anxiety.

A newspaper reports that people who drink wine with dinner have lower rates of heart disease. The report concludes that wine consumption prevents heart disease. Which of the following best identifies the flaw in this reasoning?

Answer: Wine drinkers may differ from non-wine drinkers in diet, income, or lifestyle in ways that affect heart health.

People who drink wine with dinner may have generally healthier Mediterranean diets and higher incomes—factors that independently reduce heart disease risk.

A randomized experiment tests whether a new reading program improves literacy scores. Students are randomly assigned to the program or a control group. At the end of the year, the program group scores significantly higher. Which statement is most accurate?

Answer: The reading program caused improved literacy scores in this experiment.

Random assignment supports a causal conclusion for the study sample; broader generalizability requires evidence that the sample represents the broader population.

A research team observes that neighborhoods with community gardens have lower rates of childhood obesity. A city planner proposes building community gardens to reduce childhood obesity. Which type of evidence would most strengthen the causal claim?

Answer: A randomized experiment in which neighborhoods are randomly assigned to receive a community garden or not

Only random assignment of neighborhoods to conditions can control for the many ways that neighborhoods with and without community gardens may already differ.

A study randomly selects 1,000 adults from a voter roll in one state, then randomly assigns them to receive financial literacy training or a placebo workshop. The trained group significantly increases retirement savings. Which conclusion is best supported?

Answer: Financial literacy training caused an increase in retirement savings for adults similar to those in this state.

Random assignment supports the causal claim; the voter roll sample allows generalization to adults in this state but not necessarily all adults globally.

An observational study finds that high school students who sleep more than 8 hours per night earn better grades than those who sleep fewer hours. A parent concludes that forcing teenagers to sleep 9 hours will improve their grades. What is the most important flaw in this reasoning?

Answer: The study is observational; students who sleep more may have less homework, lower stress, or other advantages that also improve grades.

In an observational study, students who sleep more may have other advantages (lighter course loads, lower stress, better home environments) that account for their better grades.