Databricks AI Engineer Case Interview: Lakehouse-Scale RAG and Model Serving

This live, scenario-based case mirrors real Databricks AI Engineer conversations: design, evaluate, and productionize an enterprise-grade LLM application on the Databricks Lakehouse. You will whiteboard an end-to-end architecture, make cost/performance tradeoffs, and dive into practical details of building, deploying, and monitoring AI systems on Databricks. What the case covers: 1) Problem framing and requirements discovery (10–15 min) - You’re supporting a Fortune 500 customer that wants a secure, low-latency internal assistant over millions of documents in cloud object storage. Clarify: data domains and formats (PDFs, HTML, parquet), privacy/PII constraints, target latency/SLOs, expected QPS, multi-cloud posture (AWS/Azure/GCP), and budget guardrails. 2) Lakehouse-centric architecture (15–20 min) - Propose a reference design using Databricks-native components: ingestion via Auto Loader into Delta Lake; bronze/silver/gold curation with Delta Live Tables or Workflows; governance, lineage, and access control with Unity Catalog; embeddings generation at scale with PySpark and MLflow tracking; vector indexing with Databricks Vector Search; LLM inference via Databricks Model Serving (serverless endpoints) and prompt orchestration/evals with MLflow. Discuss alternatives (open-source vs. Mosaic AI foundation models vs. external APIs) and justify choices. 3) Retrieval pipeline deep dive (10–15 min) - Document splitting and chunking strategy; embeddings model choice and dimensionality; index maintenance (upserts, re-embedding cadence), hybrid retrieval (vector + metadata/SQL filters), and query planning. Handle freshness, de-duplication, schema drift, and late-arriving data. Explain Delta Lake optimizations (Z-Ordering, file sizing, OPTIMIZE/VACUUM) for fast negative lookups and governance-aware retrieval. 4) Serving, reliability, and cost (10–15 min) - Low-latency design with serverless Model Serving, request batching, and dynamic scaling; caching layers (semantic/prompt/result cache) and guardrails (content filtering, PII redaction). Cover observability with MLflow metrics, request tracing, and Lakehouse-native logging; A/B tests and offline/online evals. Provide back-of-the-envelope estimates for throughput, token costs, and storage, reflecting Databricks’s truth-seeking, customer-obsessed culture. 5) Failure modes, security, and compliance (5–10 min) - Multi-tenant isolation with Unity Catalog; secrets management; data residency; rollback/blue-green for model versions; chaos scenarios (embedding drift, endpoint cold starts, index corruption) and mitigations. 6) Hands-on snippet (optional, 5–10 min if time) - Walk through a short PySpark/SQL sketch: building an embeddings table from Delta, writing to Vector Search, and calling a serving endpoint; or explain how you’d profile and optimize a slow job (cluster sizing, Photon, partitioning, shuffle reduction). Evaluation signals (Databricks-specific): - Lakehouse fluency (Delta Lake, Unity Catalog, Workflows, Model Serving, Vector Search) and pragmatic tradeoffs. - Strong system design under real constraints (latency, scale, governance) with clear cost-awareness. - Experimentation discipline (MLflow tracking/evals) and measurable success criteria. - Ownership mindset, crisp communication, and truth-seeking via estimates and data. What to expect from interviewers: - Pushback and “what-if” pivots (e.g., strict PHI/PII, on-prem data via Delta Sharing, sudden 10x scale). You’re rewarded for iterating quickly, justifying decisions, and tying choices to customer outcomes.