MS-DS Master of Data science Machine Learning 5 — Questions and Answers
Question 1: What is the key difference between bagging and boosting ensemble methods?
- Bagging uses decision trees; boosting uses linear models
- Bagging trains base learners in parallel on bootstrap samples; boosting trains them sequentially, each correcting prior errors (Correct answer)
- Bagging requires labeled data; boosting is semi-supervised
- Bagging uses soft voting; boosting always uses hard majority voting
Correct answer: Bagging trains base learners in parallel on bootstrap samples; boosting trains them sequentially, each correcting prior errors
Bagging reduces variance by averaging independently trained models on bootstrap samples, while boosting reduces bias by sequentially fitting models that focus on previously misclassified instances.
Question 2: In a convolutional neural network (CNN), what does a pooling layer do?
- Applies learned filters to detect local patterns
- Reduces spatial dimensions by aggregating features within local regions (Correct answer)
- Normalizes activations across the batch dimension
- Connects every neuron in one layer to every neuron in the next
Correct answer: Reduces spatial dimensions by aggregating features within local regions
Pooling layers (max or average) downsample feature maps spatially, reducing computational cost and providing translation invariance by summarizing local regions.
Question 3: Which evaluation metric is most appropriate for a regression model predicting house prices when outliers are present in the data?
- Mean Squared Error (MSE)
- Root Mean Squared Error (RMSE)
- Mean Absolute Error (MAE) (Correct answer)
- R-squared (R²)
Correct answer: Mean Absolute Error (MAE)
MAE is more robust to outliers than MSE/RMSE because it uses absolute differences rather than squared differences, preventing large errors from disproportionately inflating the metric.
Question 4: What problem does batch normalization solve in deep network training?
- Prevents weight matrices from becoming singular
- Reduces internal covariate shift by normalizing layer inputs to have zero mean and unit variance (Correct answer)
- Eliminates the need for activation functions in hidden layers
- Replaces dropout by acting as an implicit regularizer only
Correct answer: Reduces internal covariate shift by normalizing layer inputs to have zero mean and unit variance
Batch normalization standardizes each layer's inputs across the mini-batch, stabilizing the distribution of activations and allowing higher learning rates and faster convergence.
Question 5: In the context of Shapley values (SHAP), what does a negative SHAP value for a feature indicate?
- The feature is irrelevant and should be removed from the model
- The feature decreased the model's prediction relative to the average prediction (Correct answer)
- The feature has a negative correlation with the target variable
- The feature increases the prediction for negative class labels only
Correct answer: The feature decreased the model's prediction relative to the average prediction
A negative SHAP value means that feature's contribution pushed the model's output below the baseline (average prediction), indicating it reduced the predicted value for that instance.
Question 6: What is the purpose of the attention mechanism in transformer-based models?
- To apply positional encodings to input tokens
- To compute a weighted combination of all input positions, allowing each position to attend to relevant context globally (Correct answer)
- To normalize the hidden states after each feed-forward layer
- To reduce the dimensionality of token embeddings before classification
Correct answer: To compute a weighted combination of all input positions, allowing each position to attend to relevant context globally
Attention computes query-key similarity scores across all positions, then uses them to weight the values, enabling each token to directly access information from any other token regardless of distance.
Question 7: Which assumption of linear regression is violated when the residuals' variance increases systematically with the predicted values?
- Linearity
- Independence of errors
- Homoscedasticity (Correct answer)
- Normality of residuals
Correct answer: Homoscedasticity
Homoscedasticity requires constant variance of residuals across all levels of predictors; when variance grows with fitted values, this assumption is violated (heteroscedasticity).
MS-DS Master of Data science Machine Learning 5
What is the key difference between bagging and boosting ensemble methods?