Data Science Model Evaluation and Validation 5

Question 1

Which metric is most appropriate for evaluating a regression model?

Accepted Answer

Root Mean Squared Error (RMSE)

Answer

F1 score

Answer

AUC-ROC

Answer

Precision

Question 2

The R-squared (coefficient of determination) value represents:

Accepted Answer

The proportion of variance in the target explained by the model

Answer

The average absolute error

Answer

The number of features used

Answer

The classification accuracy

Question 3

Compared to RMSE, Mean Absolute Error (MAE) is:

Accepted Answer

Less sensitive to large outliers

Answer

More sensitive to large outliers

Answer

Only usable for classification

Answer

Always larger than RMSE

Question 4

Adjusted R-squared is preferred over R-squared when:

Accepted Answer

Comparing models with different numbers of predictors

Answer

The target is categorical

Answer

There is no validation set

Answer

Data is perfectly clean

Question 5

For a multi-class classification problem, a macro-averaged F1 score:

Accepted Answer

Averages the F1 of each class equally regardless of class size

Answer

Weights each class by its frequency

Answer

Only considers the majority class

Answer

Is identical to accuracy

Question 6

The log loss (cross-entropy) metric rewards a classifier for:

Accepted Answer

Outputting well-calibrated, confident probabilities for correct classes

Answer

Producing the fewest false positives

Answer

Maximizing the ROC area only

Answer

Having the smallest training set

Question 7

When choosing between two models, why should the comparison use the same held-out test set?

Accepted Answer

To ensure performance differences reflect the models, not different data

Answer

To make training faster

Answer

To increase the test set size

Answer

To avoid using cross-validation