MS-DS Master of Data science MS-DS Master of Data science Model Evaluation and Validation Questions and Answers 2

Question 1

In a stratified k-fold cross-validation, what is preserved in each fold compared to standard k-fold?

Accepted Answer

The proportion of each class label

Answer

The ordering of observations

Answer

The number of features selected

Answer

The random seed used for splitting

Question 2

Which metric is most appropriate for evaluating a model when the cost of false negatives is significantly higher than false positives?

Accepted Answer

Recall

Answer

Precision

Answer

Accuracy

Answer

Specificity

Question 3

What does a high variance and low bias in a model's predictions typically indicate?

Accepted Answer

The model is overfitting the training data

Answer

The model is underfitting the training data

Answer

The model has reached optimal complexity

Answer

The model needs more features

Question 4

When performing nested cross-validation, what is the purpose of the inner loop?

Accepted Answer

Hyperparameter tuning

Answer

Final model evaluation

Answer

Feature extraction

Answer

Data augmentation

Question 5

A calibration curve (reliability diagram) plots predicted probabilities against observed frequencies. What does a perfectly calibrated model's curve look like?

Accepted Answer

A diagonal line from (0,0) to (1,1)

Answer

A horizontal line at y = 0.5

Answer

An S-shaped sigmoid curve

Answer

A vertical line at x = 0.5

Question 6

Which resampling method provides the lowest variance estimate of model performance but can have pessimistic bias for small datasets?

Accepted Answer

Leave-one-out cross-validation

Answer

Bootstrap with replacement

Answer

Holdout validation with 50-50 split

Answer

Repeated random subsampling