Data Science Data Science Model Performance and Evaluation Questions and Answers 2 — Questions and Answers
Question 1: Which metric is most appropriate for evaluating a classification model when the dataset has a severe class imbalance?
- Accuracy
- Precision-Recall AUC (Correct answer)
- Mean Squared Error
- R-squared
Correct answer: Precision-Recall AUC
Precision-Recall AUC focuses on the minority class performance and is far more informative than accuracy when classes are heavily imbalanced.
Question 2: What does a high variance and low bias in a model typically indicate?
- The model is underfitting the data
- The model is overfitting the training data (Correct answer)
- The model has reached optimal performance
- The model needs more features
Correct answer: The model is overfitting the training data
High variance with low bias means the model fits training data very closely but fails to generalize, which is the hallmark of overfitting.
Question 3: In k-fold cross-validation with k=10, how many times is each data point used in the test set?
- 10 times
- 5 times
- 1 time (Correct answer)
- It depends on the dataset size
Correct answer: 1 time
In k-fold cross-validation, the data is split into k folds, and each fold serves as the test set exactly once across all iterations.
Question 4: What is the primary purpose of a calibration curve (reliability diagram) in model evaluation?
- To measure feature importance
- To assess whether predicted probabilities match actual outcome frequencies (Correct answer)
- To determine the optimal number of clusters
- To visualize training loss over epochs
Correct answer: To assess whether predicted probabilities match actual outcome frequencies
A calibration curve plots predicted probabilities against actual frequencies to reveal whether a model's confidence scores are trustworthy.
Question 5: Which evaluation approach is most suitable when you need to assess a time-series forecasting model?
- Random k-fold cross-validation
- Stratified sampling
- Walk-forward validation (Correct answer)
- Leave-one-out cross-validation
Correct answer: Walk-forward validation
Walk-forward validation respects temporal ordering by always training on past data and testing on future data, preventing data leakage.
Question 6: What does a Matthews Correlation Coefficient (MCC) of 0 indicate about a binary classifier?
- Perfect classification
- Performance no better than random guessing (Correct answer)
- The model always predicts the positive class
- The model has zero false positives
Correct answer: Performance no better than random guessing
An MCC of 0 indicates the classifier performs no better than random prediction, while +1 indicates perfect and -1 indicates total disagreement.
Data Science Data Science Model Performance and Evaluation Questions and Answers 2
Which metric is most appropriate for evaluating a classification model when the dataset has a severe class imbalance?