MS-DS Master of Data science Master of Data Science 1 — Questions and Answers
Question 1: Which ensemble learning method trains multiple models sequentially, where each model attempts to correct the errors made by the previous one?
- Bagging
- Boosting (Correct answer)
- Random Forest
- Stacking
Correct answer: Boosting
Boosting trains models sequentially, with each subsequent model focusing on the misclassified examples from the previous model. Bagging trains models independently in parallel, Random Forest is a specific bagging method, and stacking combines predictions using a meta-learner.
Question 2: In Principal Component Analysis (PCA), what do the eigenvalues of the covariance matrix represent?
- The direction of maximum variance in the data
- The amount of variance explained by each principal component (Correct answer)
- The correlation between original features
- The mean of each transformed feature
Correct answer: The amount of variance explained by each principal component
Eigenvalues represent the variance explained by each principal component. The eigenvectors (not eigenvalues) represent the directions of maximum variance. This is why eigenvalues are used to decide how many components to retain.
Question 3: What distinguishes stochastic gradient descent (SGD) from batch gradient descent?
- SGD uses a higher learning rate by default
- SGD updates model parameters using one (or a few) training examples per iteration rather than the entire dataset (Correct answer)
- SGD computes the exact gradient of the loss function
- SGD is only applicable to convex loss functions
Correct answer: SGD updates model parameters using one (or a few) training examples per iteration rather than the entire dataset
SGD updates weights using a single sample (or mini-batch) at a time, making each update faster and introducing noise that can help escape local minima. Batch gradient descent uses the full dataset for each update, computing the true gradient but at higher computational cost.
Question 4: In the context of a Master of Data Science program, what does 'data leakage' during model training refer to?
- Unauthorized access to sensitive training datasets
- When information from the test set inadvertently influences the training process (Correct answer)
- Loss of training data due to storage failures
- Overfitting caused by too many features
Correct answer: When information from the test set inadvertently influences the training process
Data leakage occurs when information outside the training data (typically from the validation or test set) is used to build the model, resulting in overly optimistic performance estimates that do not generalize. This is a critical pitfall in real-world data science workflows.
Question 5: Which probability theorem is foundational to generative classifiers like Naive Bayes, allowing posterior class probabilities to be computed from likelihoods and priors?
- Central Limit Theorem
- Law of Large Numbers
- Bayes' Theorem (Correct answer)
- Jensen's Inequality
Correct answer: Bayes' Theorem
Bayes' Theorem relates the posterior probability P(class|data) to the likelihood P(data|class) and the prior P(class). Naive Bayes classifiers apply this with the 'naive' assumption of conditional feature independence to make computation tractable.
Question 6: What is the vanishing gradient problem in deep neural networks, and which architecture feature was specifically introduced to mitigate it?
- Gradients become too large during backpropagation; mitigated by dropout layers
- Gradients shrink exponentially as they propagate to earlier layers; mitigated by residual (skip) connections (Correct answer)
- Weights converge to zero due to L2 regularization; mitigated by batch normalization
- The loss surface becomes flat near the optimum; mitigated by adaptive learning rates
Correct answer: Gradients shrink exponentially as they propagate to earlier layers; mitigated by residual (skip) connections
In deep networks, gradients computed during backpropagation are multiplied across many layers, causing them to shrink exponentially and making early-layer weights learn very slowly. Residual connections (introduced in ResNets) allow gradients to flow directly through skip connections, bypassing this multiplicative decay.
MS-DS Master of Data science Master of Data Science 1
Which ensemble learning method trains multiple models sequentially, where each model attempts to correct the errors made by the previous one?