Design a Music Click-Through Prediction Platform
Build a reproducible music-ranking feature and model pipeline that predicts click probability and measures safe product lift.
Problem statement
Design an ML platform that predicts the probability a listener will click a recommended song. Cover song and listener representations, sparse and cold-start features, training examples, model comparison, online inference, evaluation metrics, experimentation, and what happens after an observed two-percent CTR lift.
For this exercise, assume every recommendation impression is logged with position, candidate set, policy version, and later click or non-click label; serving must rank a bounded candidate set with a pinned model and feature version.
Music-catalog ingestion, audio streaming, and recommendation candidate generation are outside scope.
What to build
- The platform can construct leakage-safe labeled examples from versioned impression and click events.
- The platform can publish reproducible song, listener, context, and sparse-feature representations.
- Online serving can score a bounded song set with pinned model and feature versions plus a safe fallback.
- Operators can compare models offline, run guarded experiments, and promote or roll back a validated release.
At scale
- For this exercise, assume 100 million daily listeners, 100 million songs, and 10 billion recommendation impressions per day.
- For this exercise, score up to 1,000 candidates per request within 80 milliseconds p95 for this ranking stage.
- For this exercise, interaction features should update within five minutes and model releases occur daily.
- For this exercise, clicks are sparse, position-biased, and occasionally delayed; every prediction and experiment exposure remains reproducible for 90 days.