Model Evaluation and Validation Flashcards

Which metric is most appropriate for evaluating a regression model?

Answer: Root Mean Squared Error (RMSE)

RMSE measures average prediction error magnitude, suited to continuous targets.

The R-squared (coefficient of determination) value represents:

Answer: The proportion of variance in the target explained by the model

R² indicates how much of the target's variance the model accounts for.

Compared to RMSE, Mean Absolute Error (MAE) is:

Answer: Less sensitive to large outliers

MAE weights all errors linearly, so it is less influenced by outliers than the squared RMSE.

Adjusted R-squared is preferred over R-squared when:

Answer: Comparing models with different numbers of predictors

Adjusted R² penalizes adding predictors that don't improve the model, enabling fair comparison.

For a multi-class classification problem, a macro-averaged F1 score:

Answer: Averages the F1 of each class equally regardless of class size

Macro-averaging treats every class equally, useful when minority classes matter.

The log loss (cross-entropy) metric rewards a classifier for:

Answer: Outputting well-calibrated, confident probabilities for correct classes

Log loss penalizes confident wrong predictions and rewards accurate probability estimates.

When choosing between two models, why should the comparison use the same held-out test set?

Answer: To ensure performance differences reflect the models, not different data

A common test set isolates model quality as the only variable in the comparison.