Preparing your design workspace

Loading the problem and your latest saved attempt…

Design Real-Time Chatbot Training and Evaluation

Serve conversational answers while building a versioned, auditable data and reinforcement-learning evaluation loop.

Problem statement

Design a chatbot that answers user questions in real time and supports an iterative training and evaluation loop. The reported interview explicitly asks how to collect training data, define reinforcement-learning tasks, identify weaknesses in reward design, and mitigate them.

For this exercise, assume authenticated users hold bounded multi-turn sessions; each answer is generated under one pinned model, prompt, retrieval-policy, and reward-definition version.

A versioned reinforcement-learning task defines its observation or input, permitted action or output, episode or trajectory boundary, terminal outcomes, and linked reward definition.

The serving path records only consented or otherwise policy-authorized, purpose-limited, privacy-filtered turn evidence and explicit user feedback without blocking the response.

Authorized reviewers may accept evidence into immutable dataset versions; deletion or consent revocation prevents that evidence from entering successor datasets and revokes candidate example access without rewriting historical audit receipts.

Evaluation runs compare candidate policies against a fixed baseline, and promotion requires quality and safety thresholds across declared slices. A retry must not duplicate a turn, feedback item, training example, trajectory, or evaluation case.

Design the online serving path, evidence eligibility and authorization contracts, labeling and dataset publication, reinforcement-learning task and reward lifecycle, offline evaluation, safe promotion, rollback, and audit.

General web crawling, training foundation models from scratch, moderation-policy authoring, and building the underlying model runtime are outside scope.

What to build

  1. Serve ordered, retry-safe conversational turns under pinned model and policy versions.
  2. Capture only authorized, purpose-limited, privacy-filtered turn, outcome, and reviewed feedback evidence with stable identities and revocation state.
  3. Version each reinforcement-learning task's observation, action, trajectory boundary, terminal outcomes, and linked reward definition.
  4. Build immutable dataset and reward-definition versions and compare candidate policies reproducibly.
  5. Promote or roll back one reviewed serving version while preserving audit and slice-level quality evidence.

At scale

  • For this exercise, assume 50,000 concurrent sessions and 5,000 new turns per second at peak.
  • For this exercise, healthy requests should begin streaming within one second at p95 and complete within ten seconds at p95.
  • For this exercise, retain privacy-filtered turn evidence for 90 days and immutable reviewed dataset and evaluation metadata for one year.
  • For this exercise, accepted turns and feedback must survive a process or single-zone failure, while model unavailability returns an explicit degraded outcome.