MS-DS Master of Data science Supervised Learning Algorithms 2 — Questions and Answers
Question 1: In a support vector machine, what is the role of the kernel trick?
- It normalizes feature values to unit scale
- It maps data into a higher-dimensional space to find a linear separator (Correct answer)
- It reduces the number of support vectors needed
- It applies L2 regularization to the weight vector
Correct answer: It maps data into a higher-dimensional space to find a linear separator
The kernel trick implicitly maps input features to a higher-dimensional space where a linear decision boundary can separate classes that are non-linearly separable in the original space.
Question 2: Which loss function is most appropriate for a multi-class classification problem with mutually exclusive classes?
- Binary cross-entropy
- Mean squared error
- Categorical cross-entropy (Correct answer)
- Hinge loss
Correct answer: Categorical cross-entropy
Categorical cross-entropy measures the difference between the predicted probability distribution and the one-hot encoded true class label across all mutually exclusive classes.
Question 3: What does the 'max_depth' hyperparameter control in a decision tree?
- The minimum number of samples required to split a node
- The maximum number of features considered at each split
- The longest path from the root to a leaf node (Correct answer)
- The minimum impurity decrease required for a split
Correct answer: The longest path from the root to a leaf node
max_depth limits the longest path from the root to any leaf, controlling model complexity and preventing overfitting by restricting how deep the tree can grow.
Question 4: A logistic regression model outputs a probability of 0.3 for the positive class. With a threshold of 0.5, how is this classified?
- Positive class, because the model is uncertain
- Negative class, because 0.3 < 0.5 (Correct answer)
- Positive class, because any non-zero probability indicates presence
- It cannot be classified without knowing the prior
Correct answer: Negative class, because 0.3 < 0.5
With a decision threshold of 0.5, any predicted probability below 0.5 is assigned to the negative class, so 0.3 maps to the negative class.
Question 5: Which of the following best describes the bias-variance tradeoff in supervised learning?
- High-bias models overfit; high-variance models underfit
- High-bias models underfit the training data; high-variance models overfit it (Correct answer)
- Bias and variance are minimized simultaneously by increasing training data
- Bias measures test error; variance measures training error
Correct answer: High-bias models underfit the training data; high-variance models overfit it
High bias means the model is too simple and underfits, while high variance means the model is too complex and overfits, capturing noise in the training data.
Question 6: In k-nearest neighbors (k-NN) regression, what happens as k increases?
- The model becomes more sensitive to individual data points
- The decision boundary becomes smoother and the model becomes less complex (Correct answer)
- Training time increases exponentially
- The model ignores distant neighbors entirely
Correct answer: The decision boundary becomes smoother and the model becomes less complex
Larger k averages over more neighbors, producing smoother predictions and reducing variance, but potentially increasing bias by over-smoothing local structure.
Question 7: Which regularization technique randomly drops neurons during training to prevent overfitting in neural networks?
- L1 regularization
- L2 weight decay
- Dropout (Correct answer)
- Batch normalization
Correct answer: Dropout
Dropout randomly sets a fraction of neuron activations to zero during each training step, forcing the network to learn redundant representations and reducing co-adaptation.
MS-DS Master of Data science Supervised Learning Algorithms 2
In a support vector machine, what is the role of the kernel trick?