Model Performance and Evaluation Flashcards

Data leakage during evaluation typically causes:

Answer: Overly optimistic performance that fails in production

Leakage lets information from the test set influence training, inflating estimates that collapse in production.

When evaluating a model on time-series data, which validation strategy is appropriate?

Answer: Time-based (forward-chaining) splits

Time-series requires forward-chaining splits so the model never trains on future data.

For a multi-class classification problem, macro-averaged F1 differs from micro-averaged F1 because macro:

Answer: Treats all classes equally regardless of size

Macro-averaging computes the metric per class and averages equally, giving small classes equal weight.

Adjusting the classification threshold from 0.5 to 0.7 will most likely:

Answer: Increase precision, decrease recall

Raising the threshold makes positive predictions stricter, typically increasing precision and lowering recall.

Which statement about adjusted R-squared is correct?

Answer: It penalizes adding predictors that don't improve the model

Adjusted R-squared penalizes extra predictors that fail to improve fit, unlike plain R-squared.

Log loss (cross-entropy) heavily penalizes a model when it:

Answer: Is confidently wrong (high probability on the wrong class)

Log loss grows sharply when a model assigns high probability to an incorrect class.

Matthews Correlation Coefficient (MCC) is valued for binary classification because it:

Answer: Gives a balanced measure even with very imbalanced classes

MCC uses all four confusion-matrix cells, providing a balanced score robust to class imbalance.