DSE - Data Science Statistical Inference and Hypothesis Testing Questions and Answers — Questions and Answers
Question 1: An A/B test was conducted to determine if changing a website's call-to-action button from green to blue would increase the click-through rate. The null hypothesis stated there is no difference in the rates. The test resulted in a p-value of 0.04. If the pre-determined significance level (alpha) was 0.05, what is the correct interpretation?
- The probability that the null hypothesis is true is 4%.
- Fail to reject the null hypothesis, as the result is not statistically significant.
- Reject the null hypothesis; the evidence suggests a statistically significant difference in click-through rates. (Correct answer)
- There is a 96% probability that the blue button is more effective.
Correct answer: Reject the null hypothesis; the evidence suggests a statistically significant difference in click-through rates.
The p-value (0.04) is less than the significance level α (0.05). This means the observed result is unlikely to have occurred by random chance alone if the null hypothesis were true. Therefore, we reject the null hypothesis and conclude there is a statistically significant effect.
Question 2: In hypothesis testing, which of the following describes a Type I error?
- Failing to reject a false null hypothesis.
- Correctly rejecting a false null hypothesis.
- Rejecting a true null hypothesis. (Correct answer)
- Correctly failing to reject a true null hypothesis.
Correct answer: Rejecting a true null hypothesis.
A Type I error, also known as a 'false positive,' occurs when the null hypothesis is rejected even though it is actually true. The probability of committing a Type I error is denoted by the significance level, alpha (α).
Question 3: A data scientist calculates a 95% confidence interval for the average daily screen time of users on a new mobile app, resulting in an interval of [35.2 minutes, 39.8 minutes]. Which of the following is the most accurate interpretation of this result?
- 95% of all users have a daily screen time between 35.2 and 39.8 minutes.
- There is a 95% probability that the true average daily screen time for all users is between 35.2 and 39.8 minutes.
- If we were to repeat this sampling process many times, we would expect 95% of the calculated confidence intervals to contain the true population mean for daily screen time. (Correct answer)
- The sample mean of daily screen time has a 95% chance of being the true population mean.
Correct answer: If we were to repeat this sampling process many times, we would expect 95% of the calculated confidence intervals to contain the true population mean for daily screen time.
The 95% confidence level refers to the reliability of the estimation method, not a specific interval. It means that if we repeated the experiment or sampling many times, 95% of the intervals we create would capture the true, unknown population parameter.
Question 4: A retail company wants to test if there is a statistically significant difference in the average transaction value among customers using three different payment methods (Credit Card, Debit Card, Mobile Pay). Which statistical test is most appropriate for this analysis?
- Analysis of Variance (ANOVA) (Correct answer)
- Paired t-test
- Independent two-sample t-test
- Chi-squared test
Correct answer: Analysis of Variance (ANOVA)
ANOVA is used to compare the means of three or more independent groups. A t-test is only suitable for comparing the means of two groups. Using multiple t-tests would inflate the probability of a Type I error.
Question 5: In the context of hypothesis testing, what does the statistical power of a test represent?
- The probability of observing a result at least as extreme as the sample, assuming the null hypothesis is true.
- The probability of correctly rejecting a false null hypothesis. (Correct answer)
- The probability of incorrectly rejecting a true null hypothesis.
- The maximum p-value allowed to declare a result as statistically significant.
Correct answer: The probability of correctly rejecting a false null hypothesis.
Statistical power is the probability that the test will correctly detect a true effect when one exists. In other words, it is the probability of avoiding a Type II error (failing to reject a false null hypothesis). Power is denoted as 1 - β.
Question 6: A research team wants to test if a new employee wellness program *improves* productivity scores. The null hypothesis is that the program has no effect on productivity. The team is only interested in whether the program leads to an increase, not a decrease. Which type of test is most appropriate?
- A chi-squared test, because the outcome is categorical.
- A two-tailed test, to see if there is any difference in either direction.
- An F-test, to compare the variances of the two groups.
- A one-tailed test, because the hypothesis is directional. (Correct answer)
Correct answer: A one-tailed test, because the hypothesis is directional.
A one-tailed test is used when the alternative hypothesis specifies a direction (e.g., greater than or less than). Since the team is specifically testing for an *improvement* (an increase in scores), a one-tailed test is appropriate. A two-tailed test would be used if they were testing for *any* difference, either positive or negative.
DSE - Data Science Statistical Inference and Hypothesis Testing Questions and Answers
An A/B test was conducted to determine if changing a website's call-to-action button from green to blue would increase the click-through rate.
The null hypothesis stated there is no difference in the rates.
The test resulted in a p-value of 0.04.
If the pre-determined significance level (alpha) was 0.05, what is the correct interpretation?