Goldman Sachs AI Engineer Case Interview — Designing a Regulator‑Ready Trade Surveillance & Communications Intelligence Platform

This case simulates a Goldman Sachs (New York) AI Engineering interview focused on building an end‑to‑end, production‑grade AI system under real constraints the firm faces. Drawing from common GS interview patterns, you’ll be challenged on depth in ML/NLP/LLMs, large‑scale systems, MLOps, and risk & controls. Scenario: - Design an AI platform that flags potential market‑abuse behaviors (e.g., spoofing, layering, insider misuse) by combining: (1) low‑latency scoring on order/trade events, and (2) NLP over employee/client communications (chat, email, voice transcripts) for context. - Deliver regulator‑auditable outputs to Compliance analysts with clear explanations and case packages. Key requirements you must address (interviewers will probe deeply): 1) Problem framing & data - Data sources: OMS/EMS trade and order events (streamed via Kafka), reference/security master, surveillance alerts history, chat/email archives, voice ASR transcripts, and historical enforcement actions. - Data quality and lineage expectations; entitlements and least‑privilege access; PII handling and data retention (e‑discovery readiness). 2) Modeling approach - Hybrid design: supervised models for known patterns; anomaly detection for novel behaviors; LLM‑aided communications triage with RAG over approved internal knowledge (firm policies, historical cases). Hallucination mitigation and prompt/response safety controls. - Explainability suitable for Compliance and regulators (global feature importance + case‑level rationale; SHAP/LIME or equivalent; exemplar retrieval). - Bias/error analysis to limit analyst fatigue: optimize precision at fixed recall; champion–challenger; drift detection; canarying. 3) Systems & performance - Two paths: (A) sub‑50 ms p95 streaming inference for order events; (B) hourly/daily batch for communications with SLA to produce analyst‑ready cases by 8:00 ET. - Reference tech you may choose: Python/Java services, Kafka, Spark/Flink, feature store, GPU/CPU autoscaling on Kubernetes, model registry (e.g., MLflow), Airflow, object storage; discuss on‑prem vs VPC patterns. - Capacity planning, cost controls, observability (metrics, logs, traces), resiliency (multi‑AZ), backpressure handling, and graceful degradation. 4) MLOps, governance, and controls (Goldman emphasis) - Model Risk Management lifecycle (documentation, validation, approvals), versioning, reproducibility, and rollback. Change management with peer review and release gates. - Data and model controls: encryption in transit/at rest (KMS), secrets management, audit trails, access reviews, and guardrails for LLM usage (no external retention, redaction, policy filters). - Reg considerations: alignment with SEC/FINRA/MiFID II surveillance expectations; retention, discoverability, and regulator‑ready reporting. 5) Product thinking & stakeholder alignment - Analyst workflow integration (case UI, triage queues, feedback loop); measuring commercial impact (alert quality lift, analyst hours saved, false‑positive reduction) and non‑functional KPIs (SLOs, latency, cost per 1k events). - Communication style: crisp trade‑off narratives for senior stakeholders; defend choices under pushback. Format (reflecting GS style): - 10 min: Clarify objectives, constraints, and success metrics. - 25 min: System/ML design whiteboarding with back‑of‑the‑envelope sizing. - 15 min: MLOps & governance deep dive (risk controls, validation, deployment, rollback). - 10 min: Metrics/experimentation plan and production runbook. - 10 min: Culture fit probes (client focus, ownership, teamwork, controls mindset) and Q&A. Evaluation rubric (how GS typically assesses): - Structured thinking under time pressure; depth across ML + distributed systems. - Risk & controls fluency and regulator‑ready documentation mindset. - Practical productionization: monitoring, SLOs, incident response, and cost awareness. - Clear storytelling to non‑technical stakeholders; ability to handle pushback and defend trade‑offs. Common follow‑ups you should anticipate: - How do you reduce analyst fatigue while preserving coverage? What are your alert quality targets and thresholds? - What’s your rollback and incident playbook if drift spikes during market open? - How do you prove the LLM didn’t leak sensitive data? What guardrails and logs exist? - How do you prepare for a regulator exam on this system within two weeks?