CAP Model Deployment & Lifecycle Management Flashcards

In the CAP framework, model retirement is triggered when:

Answer: The model no longer meets defined performance thresholds and cannot be retrained to do so

A model should be retired when sustained retraining fails to restore acceptable performance, signaling fundamental concept changes.

Which governance artifact tracks who approved a model for production and under what conditions?

Answer: Model governance log / audit trail

A model governance log records approval decisions, reviewers, conditions, and timestamps required for audit and compliance purposes.

Which type of feedback loop can cause a deployed model's predictions to become self-fulfilling and degrade data quality?

Answer: Positive feedback loop (reinforcing loop)

A reinforcing feedback loop occurs when model outputs influence future inputs (e.g., a loan denial model trained on its own denials), amplifying existing biases.

Business stakeholders reviewing a deployed model's ongoing performance should primarily examine:

Answer: Key performance indicators (KPIs) tied to the original business objective

Stakeholders care about business outcomes (revenue lift, churn reduction) rather than technical model internals.

Which strategy allows immediate rollback to a previous model version if the new version causes issues in production?

Answer: Blue-green deployment

Blue-green deployment maintains two identical environments (old and new) so traffic can be instantly switched back to the old environment if problems arise.

What is the key difference between model monitoring and model observability?

Answer: Monitoring tracks predefined metrics; observability enables investigation of unknown failure modes

Monitoring alerts on known metrics, while observability provides the data richness needed to diagnose novel, unexpected model failures.