Amazon AI Engineer Case Interview – Working Backwards LLM/RAG System Design

This case simulates an Amazon-style, Bar Raiser-inclusive technical and product deep dive focused on building a customer-obsessed AI capability using the Working Backwards approach. You will write first, clarify the customer problem via a brief PRFAQ outline, and then design an end-to-end system that could realistically operate at Amazon scale using AWS-native primitives. Example prompt: "Design an LLM-powered shopping assistant for Amazon Retail that helps customers compare similar products and verify compatibility (e.g., accessories), with enterprise-grade safety/guardrails, multilingual support, and global rollout." What you’ll cover: (1) Working Backwards + PRFAQ (10 min): articulate the customer, use cases, non-goals, and success metrics; define North Star and counter-metrics (conversion lift, CTR, bad handoff rate, latency, cost/query). (2) Technical Architecture (25 min): propose LLM/RAG architecture (document ingestion, chunking, embeddings, retrieval, grounding, prompt orchestration, safety filters), online/offline feature parity, and data pathways (S3/Glue/Feature Store, Kinesis for streaming, DynamoDB/OpenSearch for low-latency retrieval, Bedrock/SageMaker for model endpoints, IAM/KMS for security). Include latency budgets (p95/p99), throughput, caching, multi-region, and fallback behavior; discuss cost controls and autoscaling. (3) Evaluation & Experimentation (15 min): offline eval (precision/recall, groundedness, toxicity), online A/B or interleaving, guardrail metrics, abuse handling, and ramp strategy with Launch Readiness and alarms (CloudWatch, canaries, SLOs, error budgets, drift detection). (4) Operational Excellence (5 min): on-call readiness, runbooks, observability, rollback, and mechanisms for continuous improvement. (5) Leadership Principles deep dive (10 min): demonstrate Customer Obsession, Dive Deep, Ownership, Bias for Action, Insist on the Highest Standards, and Deliver Results with STAR examples; expect "why this trade-off?" and cost vs. quality probing from the Bar Raiser. Style mirrors Amazon’s emphasis on clear writing, measurable outcomes, and principled trade-offs.