CAP Methodology Selection & Analytics Problem Framing Flashcards

Which evaluation metric is MOST appropriate for a highly imbalanced binary classification problem?

Answer: Area Under the ROC Curve (AUC-ROC)

AUC-ROC evaluates discrimination ability across all thresholds and is robust to class imbalance, unlike accuracy.

A CAP analyst must choose between a linear regression model and a decision tree for a continuous outcome. The BEST reason to prefer linear regression is:

Answer: It provides easily interpretable coefficients for stakeholders

Linear regression coefficients have clear directional interpretations, making results more communicable to non-technical stakeholders.

Cross-validation is used during methodology selection primarily to:

Answer: Estimate model performance on unseen data

Cross-validation provides an unbiased estimate of how well the model will generalize by repeatedly testing on held-out data folds.

When an analytics problem involves predicting the number of events in a fixed time interval, which model family is MOST appropriate?

Answer: Poisson regression

Poisson regression is designed for count outcomes representing events in a fixed time or space interval.

Which concept describes selecting the simplest model that adequately explains the data?

Answer: Parsimony

Parsimony (Occam's Razor) favors simpler models to avoid overfitting and improve generalizability.

A team debates using a neural network versus a linear model for a small dataset with 200 rows. The CAP best practice recommends:

Answer: Use the simpler linear model to avoid overfitting

Neural networks require large datasets to generalize; on small datasets a simpler linear model typically outperforms and avoids overfitting.