Supervised Learning Algorithms Flashcards

What distinguishes a generative classifier like Naive Bayes from a discriminative one like logistic regression?

Answer: Generative models the joint distribution P(x,y); discriminative models P(y|x) directly

Generative classifiers model the joint distribution while discriminative classifiers model the conditional P(y|x) directly.

Why might one-hot encoding be necessary before applying linear models to categorical data?

Answer: Because models treat numeric category codes as having ordinal magnitude

One-hot encoding prevents the model from assuming a false ordinal relationship among category integer codes.

What is the role of support vectors in an SVM?

Answer: They are the points closest to the decision boundary that define the margin

Support vectors are the boundary-defining points that determine the position and width of the margin.

When predicting a continuous target, which supervised algorithm is appropriate?

Answer: Linear regression

Linear regression predicts continuous numeric outcomes, unlike classification algorithms.

What does early stopping accomplish when training a boosted model?

Answer: It halts adding trees when validation performance stops improving to prevent overfitting

Early stopping ends training once validation error stops improving, reducing overfitting.

Which statement about precision is correct?

Answer: It measures the fraction of predicted positives that are correct

Precision is TP / (TP + FP), the fraction of positive predictions that are actually correct.

In ensemble learning, why does combining diverse weak learners often improve accuracy?

Answer: Errors of individual learners tend to cancel out when aggregated

Aggregating diverse models lets their uncorrelated errors partially cancel, improving overall accuracy.