CVS Health AI Engineer Case Interview: Prior Authorization Triage and Formulary Intelligence

What this case covers at CVS Health (Aetna Health Care Benefits + Caremark PBM + Retail/MinuteClinic context): Scenario prompt (5–7 min): You are an AI Engineer on a cross-functional team spanning Aetna clinical operations, CVS Caremark pharmacy benefit management, and MinuteClinic. Leadership asks you to design an AI solution that triages and accelerates prior authorization (PA) requests for specialty medications. The system must: (a) extract key clinical facts from unstructured notes (provider faxes, EHR excerpts), (b) align those facts to Caremark formulary rules and plan-specific PA criteria, (c) recommend an approval/denial/needs-more-info next action with rationale, and (d) surface safe, cost-effective alternatives when applicable. You will whiteboard architecture, data flows, model choices, evaluation, deployment, and risk controls consistent with CVS Health’s scale, culture, and regulatory obligations. What interviewers look for (CVS-specific focus areas): - Patient-first, compliance-by-design thinking: explicit handling of PHI/PII under HIPAA, data minimization, and auditability across Aetna/Caremark/MinuteClinic boundaries. - Enterprise-scale pragmatism: choices that work across 10K+ retail/LTC footprint and national payer populations; cost-aware cloud usage and SLA considerations for PA turnaround times. - Safety, fairness, and governance: bias checks across demographic groups, model cards, approvals via model risk management, and human-in-the-loop pharmacist/clinician review. - Collaboration in a matrixed org: alignment with pharmacists, medical directors, utilization management nurses, and PA operations; clear handoffs and escalation paths. Case tasks and discussion flow (approximate timing): 1) Problem framing and success criteria (10 min) - Define user roles (UM nurse, pharmacist, PA rep), primary outcomes (reduced PA cycle time, medical spend management, member experience), guardrails (zero harmful recommendations). - Metrics: decision coverage, precision/recall on approval suggestion, turnaround time (P50/P95), cost-of-care impact, fairness metrics across cohorts. 2) Data and knowledge sources (8–10 min) - Unstructured: provider notes, faxes, MinuteClinic visit summaries, appeal letters. - Structured: claims (ICD-10, CPT/HCPCS), Rx (NDC), eligibility, formulary tiers, step therapy rules, PA criteria per plan. - Standards: FHIR resources for problems/medications; mapping strategy and ontology management. - PHI handling: de-identification strategy for model training vs. live inference; data retention and access controls. 3) Modeling approach (12–15 min) - Text processing: clinical NLP/LLM for entity extraction (diagnoses, labs, prior treatments), retrieval-augmented generation over formulary and PA policy text; rule engine overlay for determinism on hard criteria. - Decision layer: hybrid system—LLM for rationale generation + gradient-boosted or neural classifier for action recommendation; confidence calibration with abstention. - Safety: prompt guardrails, policy-as-code for non-negotiable criteria, red-teaming for hallucinations, contraindication checks. 4) System and MLOps architecture (12–15 min) - Reference stack: message ingestion (OCR for faxes), feature store, vector store for policy retrieval, model serving with canary rollout, MLflow-style tracking, lineage for audit. - Real-time vs. async pathways; SLOs and backpressure for spikes during plan-year changes. - Monitoring: drift on language/policy updates, false escalation rates, per-plan performance dashboards; PHI access logs and immutable audit trails. 5) Risk, compliance, and human-in-the-loop (8–10 min) - Reviewer UX: structured rationale, policy snippets, and missing-info checklists; easy override and feedback capture. - Model governance: documentation, periodic revalidation, fairness reviews, rollback plan; clear separation of recommendation vs. final clinical decision. 6) Extension prompts (time-permitting) - Alternative therapy suggestions with cost/coverage context; specialty adherence outreach; integration with retail and home infusion workflows. - Handling policy drift: automated retrieval refresh, versioned embeddings, and regression tests on exemplar PA cases. Deliverables during interview: - A clear, labeled architecture sketch; reasoning for model choices; metric definitions with acceptable thresholds; privacy and safety controls; rollout plan (pilot → phased scale); and trade-offs explained in plain language. CVS Health interview style notes: - Expect structured, scenario-based probing, emphasis on ethical AI and compliance, and practical trade-offs over flashy models. Interviewers value crisp communication to non-ML stakeholders, measurable outcomes (cycle time, cost, quality), and an incremental path to production with governance checkpoints.