MS-DS Master of Data science Unsupervised Machine Learning Models 4 — Questions and Answers
Question 1: In Non-negative Matrix Factorization (NMF), what constraint separates it from standard matrix factorization methods?
- All elements in the factor matrices must be non-negative, enabling parts-based representations (Correct answer)
- The factorization must preserve the original matrix's rank exactly
- Factors must be orthogonal to one another
- The number of factors must equal the number of features
Correct answer: All elements in the factor matrices must be non-negative, enabling parts-based representations
NMF constrains all factor values to be ≥ 0, which leads to additive, parts-based decompositions (e.g., facial parts instead of eigenfaces).
Question 2: Which distance metric is most appropriate when comparing high-dimensional sparse vectors such as TF-IDF document representations?
- Cosine similarity (Correct answer)
- Euclidean distance
- Manhattan distance
- Chebyshev distance
Correct answer: Cosine similarity
Cosine similarity measures the angle between vectors and is invariant to document length, making it ideal for sparse high-dimensional text data.
Question 3: What is the primary purpose of the encoder in a Variational Autoencoder (VAE) compared to a standard autoencoder?
- The VAE encoder outputs parameters of a probability distribution rather than a fixed latent vector (Correct answer)
- The VAE encoder uses convolutional layers while standard autoencoders use fully connected layers
- The VAE encoder is trained with class labels to produce discriminative embeddings
- The VAE encoder compresses data to exactly one dimension
Correct answer: The VAE encoder outputs parameters of a probability distribution rather than a fixed latent vector
The VAE encoder predicts a mean and variance, enabling sampling from a learned latent distribution and smooth interpolation in latent space.
Question 4: In spectral clustering, what is the role of the graph Laplacian?
- It encodes pairwise connectivity, and its eigenvectors reveal the cluster structure of the data (Correct answer)
- It computes the centroid of each cluster iteratively
- It measures the density of points in epsilon-neighborhoods
- It initializes cluster assignments using random spanning trees
Correct answer: It encodes pairwise connectivity, and its eigenvectors reveal the cluster structure of the data
The eigenvectors of the graph Laplacian capture the connectivity structure of the similarity graph, allowing k-means to be applied in the spectral embedding space.
Question 5: Which criterion is commonly used to select the optimal number of topics in LDA?
- Perplexity on a held-out test corpus (Correct answer)
- Within-cluster sum of squares elbow
- Silhouette coefficient of document embeddings
- Reconstruction error of the document-term matrix
Correct answer: Perplexity on a held-out test corpus
LDA perplexity measures how well the trained model predicts unseen documents; lower perplexity on held-out data indicates a better model fit.
Question 6: What is 'concept drift' and how does it challenge unsupervised streaming clustering?
- The underlying data distribution changes over time, invalidating previously learned cluster structures (Correct answer)
- Clusters gradually merge into a single large cluster due to noise accumulation
- The algorithm loses track of cluster centroids after processing too many mini-batches
- Feature correlations become non-stationary due to hardware precision errors
Correct answer: The underlying data distribution changes over time, invalidating previously learned cluster structures
Concept drift occurs when the statistical properties of the input stream evolve, requiring streaming clustering algorithms to adapt their cluster assignments dynamically.
Question 7: What does the 'curse of dimensionality' imply specifically for distance-based clustering algorithms?
- In high dimensions, distances between points become nearly equal, making separation of clusters difficult (Correct answer)
- Clustering algorithms require exponentially more memory with each additional dimension
- High-dimensional data always has more clusters than low-dimensional data
- Distance metrics become non-metric (violate triangle inequality) in high dimensions
Correct answer: In high dimensions, distances between points become nearly equal, making separation of clusters difficult
As dimensionality increases, the ratio of maximum to minimum pairwise distances approaches 1, making it hard to identify meaningful nearest neighbors for clustering.
MS-DS Master of Data science Unsupervised Machine Learning Models 4
In Non-negative Matrix Factorization (NMF), what constraint separates it from standard matrix factorization methods?