Model Performance and Evaluation Flashcards

For a regression model, which metric penalizes large errors more heavily because it squares the residuals?

Answer: Mean Squared Error (MSE)

MSE squares residuals, so larger errors contribute disproportionately more to the total.

R-squared of 0.0 in a regression model indicates the model:

Answer: Explains no more variance than predicting the mean

An R-squared of 0 means the model does no better than always predicting the target mean.

When tuning hyperparameters, why should you use a separate validation set rather than the test set?

Answer: To avoid leaking test information and overly optimistic estimates

Tuning on the test set leaks information and inflates the final reported performance.

A model's predicted probabilities are systematically too confident. Which technique addresses this?

Answer: Probability calibration (e.g., Platt scaling)

Calibration methods like Platt scaling or isotonic regression align predicted probabilities with observed frequencies.

In the bias-variance tradeoff, a model that is too simple to capture the underlying pattern exhibits:

Answer: High bias, low variance

An overly simple model underfits, showing high bias and low variance.

Which metric is most appropriate when both false positives and false negatives carry costs and classes are imbalanced?

Answer: F1 score

F1 balances precision and recall, making it suitable for imbalanced data where both error types matter.

A precision-recall curve is generally preferred over an ROC curve when:

Answer: The positive class is rare (highly imbalanced)

PR curves are more informative than ROC curves when the positive class is rare.