Preparing your design workspace

Loading the problem and your latest saved attempt…

Design a Photo-Labeling App

Design the system behind a photo-labeling app: decide where each model runs, how GPU servers are shared, and how new model versions reach users safely.

Problem statement

People point a phone at a dog, a plant or a dish and want to know what it is. Design the system behind an app that tells them.

For this exercise, assume a phone and web app that labels everyday photos with one of about 1,000 categories (for example dog breeds, plants, foods and household objects) and shows the top three labels with a confidence for each.

The phone app can run a small model on the device; a larger, more accurate model runs on GPU servers. Users can mark a label as right or pick the correct one, and the team retrains and releases new model versions every few weeks.

Photos are private: the original photo is deleted 24 hours after it is classified unless the user has opted in to let it be used to improve the models. Detecting several objects in one photo, drawing boxes around objects, content moderation and search over past photos are outside scope.

What to build

  1. A user submits a photo and gets the top three labels with confidences; for this exercise, the server answers within 500 ms at p95 after the upload finishes.
  2. A user can mark a result as right or choose the correct label, and that feedback is kept with the result it refers to.
  3. For this exercise, the team can release a new model version gradually and roll it back, and every result records which model version produced it.
  4. For this exercise, an original photo is deleted 24 hours after classification unless the user opted in, and only opted-in photos are used for training.

At scale

  • For this exercise, assume about 5 million daily users and 20 million classifications a day, about 230 per second on average and about 1,000 per second at peak: about 900 from the phone app and 100 from the web app.
  • For this exercise, photos are up to 10 MB, and the server model takes a 384-pixel image.
  • For this exercise, one GPU serves about 200 images per second with the large model when it can batch up to 16 images.
  • For this exercise, about 2% of results receive feedback (about 400,000 a day), and about 10% of users opt in to share photos for training.