Preparing your design workspace

Loading the problem and your latest saved attempt…

Warm GPU Container Pool

Design a multi-tenant pool of prewarmed GPU containers that reduces cold starts without wasting accelerators.

Problem statement

For this exercise, assume serverless GPU workloads specify an immutable runtime image, accelerator type, region, tenant, and optional model artifact. A cold container takes about 30 seconds to become ready.

The platform predicts demand, keeps bounded warm capacity, leases containers safely, reclaims idle workers, and falls back to cold creation during misses. Exclude hardware procurement and container-runtime internals.

What to build

  1. Create, health-check, resize, and retire warm containers by compatible runtime and accelerator shape.
  2. Atomically lease one healthy container to one invocation with tenant isolation and fencing.
  3. Detect failed, leaked, or stale leases and safely return or destroy capacity.

At scale

  • Support 50 accelerator and runtime pool keys across three regions.
  • Start 95 percent of predicted invocations below 2 seconds while cold fallback may take 30 seconds.
  • Handle 10,000 lease requests per second and sudden tenfold tenant bursts.
  • Never attach one physical container to two live invocations, including during control-plane failover.