Data Science Supervised Learning Algorithms 2 — Questions and Answers
Question 1: In a decision tree, what does a higher Gini impurity at a node indicate?
- The node is more pure
- The classes are more mixed at that node (Correct answer)
- The node is a leaf
- The feature is continuous
Correct answer: The classes are more mixed at that node
Gini impurity increases as class distribution within a node becomes more mixed.
Question 2: Which supervised algorithm is most prone to overfitting if grown without depth limits?
- Naive Bayes
- An unpruned decision tree (Correct answer)
- Linear regression
- k-NN with large k
Correct answer: An unpruned decision tree
A fully grown unpruned tree can memorize training data, causing overfitting.
Question 3: What is the primary purpose of the kernel trick in support vector machines?
- To speed up gradient descent
- To enable linear separation in a higher-dimensional space without explicit mapping (Correct answer)
- To reduce the number of features
- To normalize input data
Correct answer: To enable linear separation in a higher-dimensional space without explicit mapping
The kernel trick computes inner products in a higher-dimensional space implicitly, allowing nonlinear separation.
Question 4: In logistic regression, what does the sigmoid function output represent?
- A class label directly
- A probability between 0 and 1 (Correct answer)
- The residual error
- The feature weight
Correct answer: A probability between 0 and 1
The sigmoid maps the linear combination of inputs to a probability between 0 and 1.
Question 5: Why does k-nearest neighbors require feature scaling?
- It uses gradient descent
- Distance calculations are dominated by large-range features (Correct answer)
- It assumes Gaussian features
- It needs one-hot encoding
Correct answer: Distance calculations are dominated by large-range features
Unscaled features with larger ranges disproportionately influence the distance metric.
Question 6: What does the 'naive' assumption in Naive Bayes refer to?
- Features are conditionally independent given the class (Correct answer)
- The dataset is small
- All classes are equally likely
- The model ignores prior probabilities
Correct answer: Features are conditionally independent given the class
Naive Bayes assumes features are conditionally independent given the class label.
Question 7: Which metric is most appropriate for evaluating a classifier on a heavily imbalanced dataset?
- Accuracy
- F1 score or AUC-PR (Correct answer)
- Mean squared error
- R-squared
Correct answer: F1 score or AUC-PR
F1 score or precision-recall AUC better reflect performance when classes are imbalanced than accuracy.
Data Science Supervised Learning Algorithms 2
In a decision tree, what does a higher Gini impurity at a node indicate?