Practice Test GeeksDSE - Data Science Practice Test
← All DSE Flashcard Decks

Model Evaluation and Validation Flashcards

7 cards from real DSE practice questions. Tap to flip, then mark Knew It or Still Learning — missed cards come back until you master them.

Read the first 7 Model Evaluation and Validation flashcards as text

  1. What is 'data leakage' in the context of model validation?

    Answer: Information from the test set influencing model training or evaluation

    Data leakage occurs when information from outside the training boundary (e.g., test labels or future data) influences the model, causing overly optimistic results.

  2. A model trained on hospital A data is tested on hospital B data and performs much worse. This primarily illustrates:

    Answer: Distribution shift / covariate shift

    Distribution shift occurs when the statistical properties of the test environment differ from training, leading to performance degradation.

  3. The G-mean metric for imbalanced classification is calculated as:

    Answer: √(Sensitivity × Specificity)

    G-mean is the geometric mean of sensitivity and specificity, balancing performance across both the positive and negative classes.

  4. When evaluating a multi-class classifier, 'macro-averaged' F1-score differs from 'weighted-averaged' F1-score in that macro averaging:

    Answer: Gives equal weight to all classes regardless of size

    Macro-averaging computes the metric for each class independently and takes the unweighted mean, treating all classes equally regardless of support.

  5. Which of the following is a sign of high variance (overfitting) when examining learning curves?

    Answer: Training error is low but validation error is much higher

    A large gap between low training error and high validation error is the classic signature of overfitting (high variance).

  6. What is the purpose of the McNemar test in model evaluation?

    Answer: Statistically comparing two classifiers' error rates on the same test set

    McNemar's test uses a contingency table of paired predictions to determine if two classifiers make statistically different errors on the same samples.

  7. In the context of regression evaluation, what does a residual plot where residuals fan out as fitted values increase suggest?

    Answer: Heteroscedasticity — non-constant error variance

    Residuals fanning out (increasing spread) indicate heteroscedasticity, meaning the error variance is not constant across the range of predictions.