NVIDIA AI Engineer Case Interview: GPU-Accelerated Inference Optimization and Deployment

This case simulates a realistic NVIDIA-style deep technical problem focused on building and optimizing an end-to-end AI inference pipeline on NVIDIA GPUs. It mirrors common patterns from real NVIDIA interviews, where candidates are expected to dive deep, reason quantitatively, write or sketch code, and make defensible performance trade-offs under constraints. Scenario: You are handed a PyTorch vision model (e.g., an object detector with dynamic input shapes) that currently runs at ~60 ms P95 latency on an A100 for batch=1. A partner team needs ≤20 ms P95 latency and ≥2,000 inferences/sec per node with strict numerical fidelity (mAP drop ≤0.5% from FP32). You must propose and defend an approach to: (1) optimize and deploy using NVIDIA tooling and libraries, (2) profile and remove bottlenecks, and (3) scale across multiple GPUs with predictable tail latency. What you’ll cover (focus areas): - Optimization path: FP32→FP16 and optional INT8 (PTQ with proper calibration sets), TensorRT engine building, dynamic shape handling, kernel auto-tuning, tactic selection, CUDA Graphs to reduce launch overhead, and batch/stream sizing for latency vs. throughput. - GPU fundamentals in practice: memory hierarchy and bandwidth limits, warp-level execution, occupancy, avoiding divergence, kernel fusion opportunities, and when to offload pre/post-processing to the GPU. - Tooling and libraries: TensorRT, Triton Inference Server, cuDNN/cuBLAS, CV-CUDA or DALI for preprocessing, NCCL for multi-GPU, Nsight Systems/Compute for profiling, nvidia-smi/DCGM for observability. - Deployment design: Triton model repository layout, multiple optimization profiles, instance groups, pinned-memory I/O, request batching, concurrency settings, health/metrics endpoints, warmup strategies, and MIG for latency isolation where appropriate. - Quality and reliability: numerical parity validation vs. FP32 (per-layer or end-to-end), handling dynamic shapes and corner cases, deterministic behavior requirements, and rollback strategies. - Systems thinking: single-GPU vs. multi-GPU scaling strategy (data parallel with NCCL, sharding engines, or model ensembles), node-level limits (PCIe/NVLink), and cost/perf trade-offs across A100/H100. Typical flow (NVIDIA interview style): - 10 min: Clarify objectives and constraints; propose a performance measurement plan and initial optimization roadmap (show you think in numbers, not just heuristics). - 35–40 min: Deep dive with whiteboarding and code sketching. Expect to outline Python/C++ snippets for TensorRT engine creation, Triton config (config.pbtxt), CUDA stream usage, and a brief custom CUDA or Tritor/TensorRT plugin sketch for a bottleneck op. You’ll be asked to interpret Nsight timelines, reason about SM occupancy, and justify kernel fusion or graph capture choices. - 15–20 min: Stress questions and trade-offs. Examples: When does INT8 hurt accuracy and how do you bound it? How to choose optimal batch/concurrency for P95? How to handle dynamic shapes efficiently in TensorRT? When to use MIG vs. multi-process service instances? How to fix a memory-bound op (coalescing, shared memory tiling, mixed-precision accumulation)? What interviewers evaluate (aligned with NVIDIA culture): - Depth and rigor: Evidence-based reasoning, numerical back-of-envelope estimates (roofline thinking), and precise language. - Practical GPU fluency: Correct, specific use of NVIDIA tools and APIs; ability to profile, interpret traces, and prioritize the highest-impact fixes. - End-to-end ownership: Consideration of data pipelines, deployment, observability, and rollback, not just the model. - Code quality under pressure: Clear, correct, and efficient pseudo/real code with attention to edge cases and performance. Deliverables in-session: A concrete optimization plan with expected latency/throughput impacts per step, a sketched Triton configuration, and a brief outline of profiling experiments you would run (metrics, tools, and success criteria).