Okta AI Engineer Case Interview: Designing Risk‑Based Authentication and Identity Signals at Multi‑Tenant Scale

This case mirrors Okta’s customer‑first, security‑by‑design interview style and focuses on building an AI/ML capability that augments Adaptive MFA with real‑time risk scoring and clear, admin‑facing explanations. Expect a whiteboard‑heavy, inquiry‑driven session that emphasizes pragmatic trade‑offs, tenant isolation, privacy, and operability. Format (what to expect at Okta): - 5–10 min: Clarify problem, align on customer goals for workforce and customer identity (CIAM) tenants. - 30–40 min: End‑to‑end design of data/ML and serving paths. - 10–15 min: Deep dives on safety, privacy, abuse resistance, and on‑call readiness. - 5–10 min: Metrics, iteration plan, and trade‑offs. Case prompt you’ll work through: “Design a real‑time risk scoring service that ingests Okta System Log events, device/network context, and org policy signals to predict anomalous sign‑ins and trigger step‑up challenges via Adaptive MFA. The service must be explainable to admins, privacy‑preserving across tenants, and resilient across regions.” Scope and constraints (specific to Okta’s environment): - Multi‑tenant isolation: strict org boundaries, scoped access, per‑tenant configuration, and safe defaults for new orgs. - Data sources: Okta System Log (authN/authZ events, factors, device/context signals), inline hooks/webhooks, IP intelligence, optional customer‑provided signals. - Real‑time serving: p95 < 100 ms for a risk score API on the critical sign‑in path; graceful degradation if the scorer is unavailable. - Privacy/compliance: data minimization, PII handling, retention windows, regional data residency, consent surfaces; auditability for admin actions. - Explainability: admin dashboard rationale (e.g., new device + impossible travel + high‑risk ASN) with human‑readable policies and links to Okta Expression Language equivalences. - Safety/abuse: adversarial behavior, model poisoning defenses, threshold tuning, lockout safeguards, and appeal paths. - Reliability: multi‑region failover, circuit breakers, canaries, and SLOs aligned to identity availability expectations. What a strong solution covers (how Okta evaluates): - Architecture: offline training pipeline + feature store (time‑windowed aggregates, tenant‑aware features), online inference service, and a policy orchestration layer that maps risk bands to actions (allow/step‑up/deny). - Modeling choices: simple, auditable baselines (rules, logistic regression) → ensemble/sequence models; drift detection; per‑tenant vs global models; thresholding and cost‑of‑error framing. - APIs and contracts: RiskScore(v1) request/response schema, schema versioning, idempotency, and backpressure behavior on spikes. - Experimentation: staged rollout, shadow mode, A/B tests by tenant cohort, pre‑post guardrails (helpdesk tickets, challenge rates, conversion). - Observability/on‑call: RED metrics for the service, model/feature quality monitors, dashboards/alerts, and a rollback playbook. - Security: least‑privilege access to logs/features, secrets management, audit trails, and change management. - Culture fit signals: customer empathy (impact on friction), transparency (clear trade‑offs), integrity (safe defaults), and pragmatic iteration. Stretch topics interviewers may probe: - Natural‑language ‘policy copilot’ that converts admin intent into Okta Expression Language with guardrails and human‑in‑the‑loop review. - Cross‑org threat sharing without leaking tenant data (e.g., privacy‑preserving aggregation). - Handling major incident scenarios (e.g., third‑party intel indicating active credential‑stuffing) and rapid threshold updates. Deliverables during the session: - A diagram of training/inference paths, data boundaries, and failure modes. - A concise API spec and data contract for features. - A metrics plan (risk precision/recall, step‑up success rate, false‑positive impact, latency/SLOs) and a safe rollout plan. - Clear trade‑offs you would make in the first 90 days vs. longer‑term hardening.