Wells Fargo AI Engineer Case Interview: Responsible AI for Fraud Detection under Model Risk Governance

Overview: This Wells Fargo case simulates end-to-end design of a real-time card fraud detection service at a highly regulated bank. Candidates must balance model performance with governance, risk, and controls aligned to Wells Fargo model risk management, responsible AI, and customer experience standards. What the case covers: - Problem framing: Define objectives such as reducing fraud losses while minimizing customer friction. Translate business goals into technical metrics and guardrails. Propose success measures like cost-weighted recall and precision at a fixed false positive rate, ROC AUC, latency SLOs, and availability targets. - Data and privacy: Propose data sources such as transaction stream, device and network signals, merchant attributes, customer tenure and history. Address PII handling, data minimization, deidentification or tokenization, encryption in transit and at rest, RBAC, data retention, and lineage cataloging to meet audit expectations. - Modeling approach: Outline baseline and challenger models, for example gradient boosted trees with calibrated probabilities and a deep learning challenger. Handle severe class imbalance via stratified sampling, cost-sensitive learning, or focal loss. Engineer time-windowed features, velocity features, graph features, and geo consistency. Specify explainability methods such as SHAP for per-decision reasons to support operations and model validation. - Fairness and responsible AI: Identify potential proxy bias sources and propose monitoring by segment. Define fairness checks appropriate for credit or fraud contexts such as false positive parity bands and equal opportunity analysis where applicable, while ensuring lawful, policy-aligned use of attributes. Describe mitigations and documentation of limitations. - Governance and controls: Map your design to model risk management expectations including development documentation, independent validation, challenge process, change management, and versioned approvals. Include champion challenger strategy, stability monitoring such as PSI thresholds, backtesting, and an audit-ready decision log. - System design and MLOps: Propose the online scoring architecture with streaming ingestion, feature computation consistency between training and serving, model registry, CI CD with approvals, canary and rollback plans, and incident runbooks. Include SLIs SLOs for latency and throughput, and plans for delayed label feedback in fraud. - Monitoring and operations: Define dashboards and alerts for data drift, performance decay, spike in declines, and anomaly rates by segment. Include a human-in-the-loop workflow for high-risk transactions, dispute handling, and a feedback loop to retrain. Provide thresholds and playbooks. - Security and compliance: Address secrets management, least privilege, third party risk, and secure SDLC practices. Call out logging requirements, tamper resistance, and access reviews suitable for internal audit. - Communication: Deliver an executive-ready rationale explaining tradeoffs between loss reduction and customer experience, with a clear risk acceptance matrix and go no-go criteria that would satisfy product, risk, and technology stakeholders. Candidate deliverables during the session: - Architecture sketch of streaming and batch components with data flow and control points. - Modeling plan with features, algorithms, evaluation protocol, and fairness checks. - Governance map listing required documents, sign-offs, and validation artifacts. - Rollout and monitoring plan including on-call and escalation paths. Interviewer prompts and follow ups: - How would you tune the decision threshold to hit a target customer false positive rate while meeting loss reduction goals during holiday spikes - A regulator asks for evidence supporting a specific decline. Describe the end-to-end trace you would produce and who owns each artifact. - Your drift detector fires on merchant category shift after a new promotion. What actions do you take and how do you prevent churn from false declines - The business requests adding a generative AI assistant for fraud analysts to summarize cases from internal notes. Outline retrieval boundaries, safety guardrails, and logging to satisfy privacy and audit requirements. Evaluation rubric aligned to Wells Fargo culture and standards: - Technical depth and practicality 30 percent: Sound model and data design, realistic latency and throughput estimates, consistency between training and serving. - Risk and compliance mastery 30 percent: Clear mapping to model risk management expectations, explainability, fairness, documentation, and control gates. - Communication and stakeholder alignment 20 percent: Concise tradeoff articulation, customer impact awareness, and ability to brief risk audit partners. - Monitoring and operational excellence 10 percent: Robust SLOs, drift and bias monitoring, runbooks, and champion challenger plan. - Collaboration and values fit 10 percent: Customer centric mindset, accountability for controls, and inclusive problem solving consistent with Wells Fargo team culture. Format guidance for interviewers: Begin with a 5 minute brief, 35 to 45 minutes for candidate led solutioning and whiteboarding, 10 to 15 minutes of deep dive challenges on governance and fairness, and a 5 minute wrap up on tradeoffs and risks. Provide anonymized data schema snippets or a short prompt pack if running a paired exercise. Notes: The case emphasizes real world constraints common at large US banks such as auditability, explainability, fairness, and disciplined change management. Veterans and candidates with regulated domain experience should feel free to reference parallel controls from prior roles, mapping them to the case scenario.