MS-DS Master of Data science Unsupervised Learning Techniques 4 — Questions and Answers
Question 1: What problem does the k-means++ initialization scheme address compared to random initialization?
- It eliminates the need to specify k in advance
- It reduces the chance of poor convergence to local optima by spreading initial centroids (Correct answer)
- It guarantees finding the global optimum cluster assignment
- It accelerates convergence by using gradient descent instead of Lloyd's algorithm
Correct answer: It reduces the chance of poor convergence to local optima by spreading initial centroids
k-means++ selects each subsequent initial centroid with probability proportional to its squared distance from the nearest existing centroid, leading to better spread and faster convergence.
Question 2: In spectral clustering, what is the graph Laplacian used for?
- To compute pairwise Euclidean distances between data points
- To embed data into a low-dimensional space where standard clustering is applied (Correct answer)
- To determine the optimal number of clusters automatically
- To normalize features before constructing the similarity graph
Correct answer: To embed data into a low-dimensional space where standard clustering is applied
Spectral clustering computes the eigenvectors of the graph Laplacian to embed the data into a space that reveals cluster structure, then applies k-means on this embedding.
Question 3: Which of the following is a core limitation of PCA for dimensionality reduction in the context of unsupervised learning?
- PCA cannot handle datasets with more features than samples
- PCA only captures linear relationships and misses nonlinear structure in the data (Correct answer)
- PCA requires specifying the number of components before seeing the data
- PCA is computationally infeasible for datasets with fewer than 1,000 dimensions
Correct answer: PCA only captures linear relationships and misses nonlinear structure in the data
PCA finds orthogonal directions of maximum variance using linear projections, so it cannot capture curved manifolds or other nonlinear structure in high-dimensional data.
Question 4: A data scientist applies k-means to customer transaction data and observes that the algorithm assigns all points to one cluster in the first iteration. What is the most likely cause?
- The number of clusters k is too large relative to the data size
- All initial centroids were placed in the same region of the feature space (Correct answer)
- The data contains too many categorical features for k-means to handle
- The convergence threshold was set too loosely
Correct answer: All initial centroids were placed in the same region of the feature space
If all centroids are initialized very close together, all points become nearest to one centroid, causing a degenerate solution; k-means++ initialization mitigates this.
Question 5: In autoencoders used for anomaly detection, what serves as the anomaly score for a data point?
- The activation magnitude in the bottleneck layer
- The reconstruction error between input and decoder output (Correct answer)
- The gradient norm of the loss with respect to the input
- The KL divergence between encoder output and prior distribution
Correct answer: The reconstruction error between input and decoder output
An autoencoder trained on normal data learns to reconstruct normal patterns well; anomalies have high reconstruction error because the model cannot encode and decode them accurately.
Question 6: Which statement best describes the difference between hard and soft clustering?
- Hard clustering uses distance metrics; soft clustering uses probability models
- Hard clustering assigns each point to exactly one cluster; soft clustering assigns fractional memberships (Correct answer)
- Soft clustering always produces fewer clusters than hard clustering
- Hard clustering is deterministic; soft clustering always uses random initialization
Correct answer: Hard clustering assigns each point to exactly one cluster; soft clustering assigns fractional memberships
In hard clustering (e.g., k-means), each point belongs to exactly one cluster; in soft clustering (e.g., GMM), each point has a probability of belonging to each cluster.
Question 7: What is the primary difference between agglomerative and divisive hierarchical clustering?
- Agglomerative clustering requires specifying k; divisive does not
- Agglomerative starts with each point as its own cluster and merges; divisive starts with one cluster and splits (Correct answer)
- Divisive clustering always produces a balanced dendrogram; agglomerative does not
- Agglomerative clustering only supports Euclidean distance; divisive supports any metric
Correct answer: Agglomerative starts with each point as its own cluster and merges; divisive starts with one cluster and splits
Agglomerative (bottom-up) begins with n singleton clusters and iteratively merges the most similar pair, while divisive (top-down) begins with all points in one cluster and recursively splits.
MS-DS Master of Data science Unsupervised Learning Techniques 4
What problem does the k-means++ initialization scheme address compared to random initialization?