Data Science Model Performance and Evaluation 5

Question 1

Stratified k-fold cross-validation differs from standard k-fold by:

Accepted Answer

Preserving the class distribution in each fold

Answer

Using larger folds

Answer

Only training once

Answer

Shuffling features instead of rows

Question 2

Specificity measures a classifier's ability to correctly identify:

Accepted Answer

Actual negatives

Answer

Actual positives

Answer

False positives

Answer

Predicted positives

Question 3

If two models have identical accuracy but different ROC-AUC, the higher-AUC model is better at:

Accepted Answer

Ranking positives above negatives across thresholds

Answer

Minimizing training time

Answer

Reducing feature count

Answer

Calibrating probabilities exactly

Question 4

A learning curve where both training and validation error remain high and close together indicates:

Accepted Answer

Underfitting (high bias)

Answer

Overfitting

Answer

Data leakage

Answer

Perfect generalization

Question 5

Cohen's Kappa is preferred over raw accuracy because it:

Accepted Answer

Accounts for agreement expected by chance

Answer

Ignores false negatives

Answer

Only works for regression

Answer

Requires balanced classes

Question 6

When reporting model performance, why is a confidence interval on the metric valuable?

Accepted Answer

It quantifies uncertainty in the estimate due to finite test data

Answer

It guarantees the model is unbiased

Answer

It increases the metric value

Answer

It replaces the need for a test set

Question 7

Nested cross-validation is used primarily to:

Accepted Answer

Obtain an unbiased performance estimate while tuning hyperparameters

Answer

Speed up training

Answer

Eliminate the need for a validation set forever

Answer

Increase the dataset size