Risk Assessment & Management Flashcards

What does 'sensitivity analysis' reveal in the context of ML model risk assessment?

Answer: How much model outputs change in response to small perturbations in input features

Sensitivity analysis quantifies which input features most strongly influence predictions, highlighting where input errors or manipulations pose the greatest risk to output reliability.

Which risk is most directly addressed by implementing differential privacy during ML model training?

Answer: Leakage of individual-level information from training data through model outputs

Differential privacy adds calibrated noise during training so that the model's parameters and outputs reveal negligible information about any single training record.

An ML team discovers that their model's performance is significantly worse for a minority demographic group. Which risk dimension does this primarily implicate?

Answer: Fairness and disparate impact risk

Disparate impact risk arises when a model produces systematically different—and typically worse—outcomes for protected demographic groups, creating legal and ethical exposure.

What is the key purpose of a 'threshold analysis' in ML risk management for classification models?

Answer: Evaluating how different decision thresholds trade off false positives and false negatives to match risk tolerance

Threshold analysis explores the precision-recall or FPR-TPR trade-off across all classification cut-points so risk managers can select the threshold that aligns with acceptable error costs.

Which scenario most clearly illustrates 'model dependency risk' in a production ML system?

Answer: Downstream business processes that break when the model's output schema or score range changes unexpectedly

Model dependency risk occurs when consuming systems are tightly coupled to a model's output format or range, so any undocumented change propagates failures through the entire pipeline.

During model validation, a back-testing exercise is performed on historical data. What limitation must risk managers acknowledge about this approach?

Answer: Historical data reflects past conditions and cannot guarantee future performance, especially after structural changes

Back-testing is limited because it evaluates the model on conditions that already occurred; regime changes, novel events, or structural breaks mean past performance may not predict future behavior.

A model governance framework mandates 'model tiering.' What does this typically involve?

Answer: Classifying models by their risk level to apply proportionally rigorous review and controls

Model tiering assigns risk levels (e.g., high, medium, low) based on materiality and impact so that oversight rigor—validation depth, monitoring frequency, approval authority—scales with potential harm.