MS-DS Master of Data science Machine Learning 2 — Questions and Answers
Question 1: Which ensemble method trains each successive tree to correct the residual errors of the previous trees?
- Random Forest
- Gradient Boosting (Correct answer)
- Bagging
- Stacking
Correct answer: Gradient Boosting
Gradient Boosting fits each new learner to the residuals (negative gradients) of the ensemble so far, progressively reducing prediction error.
Question 2: In the bias-variance tradeoff, which condition describes a model that performs well on training data but poorly on unseen data?
- High bias, low variance
- Low bias, high variance (Correct answer)
- High bias, high variance
- Low bias, low variance
Correct answer: Low bias, high variance
Low bias means the model fits training data well, but high variance means it captures noise and generalizes poorly to new data — the classic overfitting scenario.
Question 3: What is the primary purpose of the kernel trick in Support Vector Machines?
- To reduce the number of support vectors
- To implicitly map data to a higher-dimensional space without computing the transformation explicitly (Correct answer)
- To normalize input features before training
- To penalize misclassified points with a hinge loss
Correct answer: To implicitly map data to a higher-dimensional space without computing the transformation explicitly
The kernel trick computes dot products in a high-dimensional feature space implicitly, enabling SVMs to find nonlinear decision boundaries without the computational cost of explicit transformation.
Question 4: An ROC curve plots True Positive Rate against which metric?
- Precision
- False Positive Rate (Correct answer)
- F1 Score
- Negative Predictive Value
Correct answer: False Positive Rate
The ROC curve plots TPR (sensitivity) on the y-axis versus FPR (1 − specificity) on the x-axis across all classification thresholds.
Question 5: Which regularization technique randomly drops neurons during training to prevent co-adaptation?
- L2 regularization
- Batch normalization
- Dropout (Correct answer)
- Early stopping
Correct answer: Dropout
Dropout randomly sets a fraction of neuron activations to zero during each training step, forcing the network to learn redundant representations.
Question 6: In k-means clustering, what does the algorithm minimize?
- Total between-cluster variance
- Within-cluster sum of squared distances to centroids (Correct answer)
- Silhouette score across all points
- Davies-Bouldin index
Correct answer: Within-cluster sum of squared distances to centroids
K-means minimizes the within-cluster sum of squared Euclidean distances (inertia) between each point and its assigned cluster centroid.
Question 7: Which activation function is most commonly used in the hidden layers of modern deep neural networks due to its ability to mitigate vanishing gradients?
- Sigmoid
- Tanh
- ReLU (Correct answer)
- Softmax
Correct answer: ReLU
ReLU (Rectified Linear Unit) outputs zero for negative inputs and the identity for positive ones, providing non-saturation in the positive region and helping gradients flow during backpropagation.
MS-DS Master of Data science Machine Learning 2
Which ensemble method trains each successive tree to correct the residual errors of the previous trees?