Pinterest Data Analyst Case: Homefeed Save Rate Diagnosis & Experiment Design

This Pinterest-specific case simulates a real product analytics deep-dive on Homefeed, reflecting how Pinterest assesses problem framing, SQL/analytics rigor, experiment design, and communication. You will investigate a change in core inspiration metrics and propose data-driven next steps aligned with Pinterest’s mission of helping Pinners discover and act on ideas. Scenario - Context: One week after a Homefeed ranking update intended to increase content diversity, US Save Rate on Homefeed is down 5% week-over-week, while Closeup Rate is up 2%. Shopping-related clicks are flat. Leadership wants a fast, principled readout and a plan. - Surfaces/terms used: Homefeed impressions, Closeups (pin detail views), Saves (the action formerly called repin), Outbound Clicks, Pinner (user), Creator, Category (e.g., Food & Drink, Home Decor), Shoppable/Shopping Pins. Data you can assume (simplified interview schema) - events.pin_impressions(pinner_id, pin_id, ts, surface, country, device) - events.pin_closeups(pinner_id, pin_id, ts, surface) - events.pin_saves(pinner_id, pin_id, ts, surface) - events.outbound_clicks(pinner_id, pin_id, ts, surface, is_shopping) - dim.pinner(pinner_id, account_age_days, locale, is_creator) - dim.pin(pin_id, category, is_shoppable) - exp.assignments(pinner_id, experiment_id, variant, ts) What the case covers (focus areas) 1) Metric clarity and Pinterest context - Define precise metrics and denominators: Save Rate (saves/impressions or saves/closeups—justify choice), Closeup Rate (closeups/impressions), CTR (outbound_clicks/closeups), Saves per Active Pinner, session-level inspiration signals. - Identify guardrails aligned to inspiration and well-being: content diversity, freshness, content safety flags rate, session depth, time-to-first-save, creator impact, and ads/shopping neutrality where applicable. 2) Root-cause exploration - Propose a slicing plan: device, country, new vs. tenured Pinners (e.g., <30 vs. >180 days), category mix, shoppable vs. non-shoppable, time-of-day, content source (following vs. recommendations). - Distinguish mix-shift vs. per-item quality effects; check seasonality and novelty burnout. 3) Querying and analysis (outline-level SQL expected) - Example prompt: Compute week-over-week Save Rate on Homefeed for US by device and pinner tenure; attribute change to mix vs. rate using a decomposition (e.g., by-category fixed mix vs. actual mix). - Join exp.assignments for treatment/control breakdown if the model change was bucketed; calculate lift and CIs, and call out power/variance concerns. 4) Experiment design and decision - If not yet fully launched: propose an A/B test randomized at pinner_id for 2 weeks. Primary metric: Save Rate on Homefeed. Secondary: Closeup Rate, Saves per Active Pinner, outbound shopping CTR. Guardrails: content safety rate, content diversity (e.g., HHI/category coverage), ads revenue neutrality. Define MDE assumptions, power (80%), and stopping rules; outline bucketing hygiene and novelty effects. 5) Recommendations & next steps - Present a crisp decision: rollback, iterate, or proceed with guardrails. Suggest targeted mitigations (e.g., re-weighting for new Pinners, category cap, freshness boost) and a follow-up analysis plan. Evaluation rubric (what interviewers look for) - Problem framing and metric rigor (30%): clear definitions, Pinterest-appropriate guardrails, awareness of inspiration vs. pure clicks. - Analysis/SQL reasoning (30%): correct joins/filters, deduplication, denominator choices, sensible slices, basic variance awareness. - Experimentation (25%): valid design, MDE/power thinking, bias checks (carryover, seasonality), interpretation of lift and CIs. - Communication and product sense (15%): structured narrative, trade-offs, ties to mission and Pinner experience. Format & style expectations - Think aloud, clarify assumptions early, prefer “Pinner” and “Save” terminology, and tie recommendations back to inspiration. Light pseudocode SQL is welcome; exact syntax is not required.