MS-DS Master of Data science Statistical Inference Concepts Questions and Answers — Questions and Answers
Question 1: A data scientist at an e-commerce company runs an A/B test on a new checkout button design. The null hypothesis is that the new button has no effect on the conversion rate. After analyzing the results, they obtain a p-value of 0.03. Assuming a significance level (alpha) of 0.05, what is the correct interpretation of this p-value?
- There is a 97% probability that the new button is effective.
- There is a 3% probability that the null hypothesis is true.
- The new button caused a 3% increase in the conversion rate.
- If the null hypothesis were true, there would be a 3% probability of observing a difference in conversion rates at least as extreme as the one detected. (Correct answer)
Correct answer: If the null hypothesis were true, there would be a 3% probability of observing a difference in conversion rates at least as extreme as the one detected.
The p-value is the probability of observing the collected data, or something more extreme, assuming the null hypothesis is true. A p-value of 0.03 means that if there were truly no difference between the buttons (H0 is true), there would only be a 3% chance of seeing a result this strong or stronger just due to random sampling variation. Since 0.03 is less than the alpha of 0.05, the result is statistically significant, and we reject the null hypothesis.
Question 2: A data scientist calculates a 95% confidence interval for the average user session duration to be [150, 170] seconds. Which of the following is the correct interpretation of this interval?
- If we were to repeat this sampling process many times, we would expect 95% of the calculated confidence intervals to contain the true population mean session duration. (Correct answer)
- There is a 95% probability that the true population mean session duration is between 150 and 170 seconds.
- 95% of all individual user session durations fall between 150 and 170 seconds.
- The sample mean has a 95% chance of being the true population mean.
Correct answer: If we were to repeat this sampling process many times, we would expect 95% of the calculated confidence intervals to contain the true population mean session duration.
A 95% confidence interval relates to the reliability of the estimation procedure, not a specific interval. It means that if the same sampling method were used to generate many different samples and a confidence interval were calculated for each, about 95% of these intervals would capture the true, unknown population parameter. Common misinterpretations include stating there's a 95% probability the true mean is in the specific interval or that 95% of the data points lie within the interval.
Question 3: A pharmaceutical company conducts a clinical trial for a new drug. The null hypothesis (H0) is that the drug has no effect. The alternative hypothesis (H1) is that the drug has a positive effect. They conclude the drug is effective (reject H0), but in reality, it is not. In the context of hypothesis testing, what type of error has been made?
- Type II Error
- Standard Error
- Type I Error (Correct answer)
- Sampling Error
Correct answer: Type I Error
A Type I error occurs when the null hypothesis is incorrectly rejected when it is actually true. In this scenario, the null hypothesis (the drug has no effect) was true, but the company concluded it was effective, thereby rejecting a true null hypothesis. This is also known as a 'false positive'.
Question 4: Which of the following statements best describes the primary utility of the Central Limit Theorem (CLT) for a data scientist conducting statistical inference?
- It guarantees that any dataset will follow a normal distribution if the sample size is large enough.
- It allows for the use of statistical tests assuming a normal distribution for the sample mean's sampling distribution, even if the population distribution is non-normal, given a sufficiently large sample size. (Correct answer)
- It states that the standard deviation of a sample will always equal the standard deviation of the population.
- It is used to eliminate all sources of sampling error in an experiment.
Correct answer: It allows for the use of statistical tests assuming a normal distribution for the sample mean's sampling distribution, even if the population distribution is non-normal, given a sufficiently large sample size.
The Central Limit Theorem states that the sampling distribution of the sample mean will be approximately normally distributed, regardless of the shape of the original population distribution, as long as the sample size is sufficiently large (often cited as n > 30). This is crucial because many common inferential tests (like t-tests and the construction of confidence intervals for means) rely on the assumption of normality. The CLT provides the justification for using these tests on data from non-normal populations.
Question 5: In the context of designing an experiment, what does the 'statistical power' of a hypothesis test represent?
- The probability of making a Type I error, also known as alpha.
- The minimum sample size required to achieve a statistically significant result.
- The effect size of the observed phenomenon.
- The probability of correctly rejecting the null hypothesis when it is in fact false. (Correct answer)
Correct answer: The probability of correctly rejecting the null hypothesis when it is in fact false.
Statistical power is the probability of detecting an effect if there is a real effect to be detected. It is defined as 1 - β (the probability of a Type II error). A test with high power has a high probability of correctly finding a true effect and rejecting a false null hypothesis.
Question 6: A data scientist has a sample of 100 customer satisfaction scores. The distribution of these scores is unknown and appears to be heavily skewed. They need to estimate the 95% confidence interval for the median satisfaction score. Which statistical technique is most appropriate?
- Bootstrapping (Correct answer)
- Two-sample t-test
- Analysis of Variance (ANOVA)
- Chi-squared test
Correct answer: Bootstrapping
Bootstrapping is a non-parametric resampling method ideal for estimating properties of an estimator (like the median) when the underlying distribution of the data is unknown or non-normal. It involves repeatedly drawing samples with replacement from the original sample to create an empirical sampling distribution, from which a confidence interval can be derived. T-tests, ANOVA, and Chi-squared tests are inferential tests used for other purposes (comparing means, comparing variances, and analyzing categorical data, respectively) and are not suitable for this specific task.
MS-DS Master of Data science Statistical Inference Concepts Questions and Answers
A data scientist at an e-commerce company runs an A/B test on a new checkout button design.
The null hypothesis is that the new button has no effect on the conversion rate.
After analyzing the results, they obtain a p-value of 0.03.
Assuming a significance level (alpha) of 0.05, what is the correct interpretation of this p-value?