Preparing your design workspace

Loading the problem and your latest saved attempt…

Multi-Tenant GPU Cluster Scheduler

Design a topology-aware scheduler for distributed GPU jobs with gang placement, fairness, preemption, and fragmentation control.

Problem statement

For this exercise, assume customers submit training and inference jobs requesting one to thousands of GPUs plus CPU, memory, accelerator model, NVLink domain, InfiniBand fabric, rack, and rail constraints. Distributed jobs use gang scheduling.

The scheduler enforces tenant shares and reservations, supports bounded preemption, avoids stranded fragments, and recovers from node or scheduler failure. Exclude firmware, the training framework, and physical network construction.

What to build

  1. Validate and durably queue a job against quota, reservation, topology, and priority policy.
  2. Atomically reserve a topology-compatible GPU gang while balancing fairness and fragmentation.
  3. Checkpoint or terminate eligible work, fence allocations, and recover failed nodes or scheduler shards.

At scale

  • Manage 100,000 GPUs across 50,000 nodes and several accelerator generations.
  • Schedule 10,000 queued jobs while large gangs may request 1,024 GPUs.
  • Place eligible small jobs within 5 seconds at p95 and provide bounded wait estimates for reserved large jobs.
  • Never expose one GPU or MIG slice to two live allocations, even during scheduler failover.