Evaluating Statistical Claims: Observational Studies and Experiments Flashcards

An observational study tracks 3,000 adults for 10 years and finds that those who eat more vegetables have lower rates of cardiovascular disease. Researchers conclude that eating vegetables prevents cardiovascular disease. Which statement best identifies the flaw?

Answer: People who eat more vegetables may also exercise more, smoke less, and manage stress better, making it impossible to isolate the effect of vegetables alone.

In an observational study, vegetable consumption is confounded with other healthy lifestyle behaviors, making it impossible to attribute lower cardiovascular disease rates solely to vegetables.

A randomized experiment assigns 100 employees to use a productivity app and 100 to continue without it for 3 months, then measures tasks completed. App users complete significantly more tasks. Which conclusion is best supported?

Answer: Using the productivity app caused an increase in tasks completed for employees in this study.

Random assignment supports the causal claim that the app caused improved productivity within this study's timeframe and participant group.

A study finds that cities that host major sporting events have higher retail sales in the month of the event. A city official concludes that hosting events causes increased retail sales. What is the most important limitation?

Answer: The association may be confounded by other factors, such as the events being held during peak tourist seasons or economic booms.

Events are often held during favorable economic periods or tourist seasons, creating a confound that could explain the correlation without the events themselves causing increased sales.

Researchers randomly assign 60 nursing home residents to receive weekly social visits and 60 to receive standard care for 6 months, then measure cognitive function. The visited group shows significantly better cognitive scores. What does random assignment allow researchers to conclude?

Answer: Social visits caused improved cognitive function in these nursing home residents.

Random assignment to social visits vs. standard care supports a causal conclusion that social visits improved cognitive scores for this experimental sample.

An observational study of 4,000 middle school students finds that those who participate in sports have lower rates of obesity. A physical education teacher concludes that requiring all students to play sports will reduce obesity. What is the most significant concern?

Answer: Students who participate in sports may also engage in more physical activity outside school and have more active families, confounding the relationship.

Student athletes likely differ from non-athletes in many health-related behaviors (general activity levels, diet, family norms), creating confounding in this observational study.

A research team wants to test whether a new anti-anxiety therapy works. They randomly select 300 patients from anxiety clinics across 5 states, then randomly assign them to the new therapy or standard therapy. The new therapy group shows significantly lower anxiety scores. What can the researchers most validly conclude?

Answer: The new therapy caused lower anxiety and the results likely generalize to patients at similar anxiety clinics.

Random selection from anxiety clinics across 5 states supports generalization to similar clinic patients; random assignment supports the causal claim.

A study reports that children raised in bilingual households outperform monolingual children on executive function tasks. A researcher claims bilingualism directly improves executive function. Which observational confound is most likely?

Answer: Bilingual households may differ from monolingual households in cultural expectations, cognitive stimulation, or parent education levels.

Bilingual households often have other characteristics—such as higher parental education or cultural emphasis on cognitive skills—that independently enhance executive function.