GPU Context-Switching Software Stack
Design a software stack that schedules GPU contexts, preserves execution state and routes asynchronous completions.
Problem statement
For this exercise, assume one GPU execution engine serves several applications through independent contexts. Design the software stack for submitting command buffers, priority scheduling and preemption, switching contexts, saving/restoring execution state and returning results to the correct application.
Hardware provides asynchronous tagged safe-point, state-save and restore operations; safe points occur at most 5 ms apart and a typical save/restore costs about 1 ms.
Device buffers remain resident under their owning context, while saved execution state includes register/checkpoint state, command progress and an address-space reference. Hardware circuitry, multi-engine coordination and transparent replay of external side effects are out of scope.
A reset may explicitly fail in-flight commands.
What to build
- Applications can create contexts, submit commands and inspect each command outcome.
- Waiting applications make progress through priority scheduling and preemption, while paused contexts can later resume with their execution state and command buffers intact.
- Execution, memory references and completions remain associated with the context lifetime that owns them.
- An interrupted or unacknowledged switch produces a reconciled result or explicit bounded failure.
At scale
- For this exercise, assume up to 32 resident contexts and 1000 submitted commands per second at moderate load.
- For this exercise, target normally starting interactive work within 20 ms at moderate load, while preserving background progress; admission may be deferred or rejected under overload.
- For this exercise, hardware safe points are at most 5 ms apart and a typical save/restore costs about 1 ms; large device buffers are not copied for each switch.