Model Evaluation and Validation Flashcards
7 cards from real Data Science practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.
Read the first 7 Model Evaluation and Validation flashcards as text
What is the primary purpose of k-fold cross-validation?
Answer: To obtain a more reliable performance estimate by averaging over multiple train/test splits
K-fold averages performance across k splits, reducing variance from any single split.
In 5-fold cross-validation, what fraction of data is used for testing in each fold?
Answer: About 20%
With 5 folds, each test fold is 1/5 (20%) while 80% trains the model.
When should stratified k-fold cross-validation be preferred?
Answer: When the dataset has imbalanced class distributions
Stratified folds preserve class proportions in each fold, important for imbalanced classes.
Leave-one-out cross-validation (LOOCV) uses how many folds?
Answer: As many folds as there are data points
LOOCV sets k equal to n, testing on one sample at a time.
Why must time-series data use specialized cross-validation like forward chaining?
Answer: Random shuffling would leak future information into training
Temporal order must be preserved so the model never trains on future data to predict the past.
A key drawback of LOOCV compared to 10-fold cross-validation is:
Answer: High computational cost since the model is trained n times
LOOCV trains n models, which is expensive for large datasets.
The validation set in a train/validation/test split is primarily used for:
Answer: Tuning hyperparameters and model selection
The validation set guides hyperparameter tuning while the test set gives the final estimate.