Design a Kubernetes-Managed Kafka Service
Design a managed control plane that provisions, scales, upgrades, and recovers isolated Kafka clusters on Kubernetes while keeping client endpoints stable.
Problem statement
For this exercise, assume customers request dedicated managed Kafka clusters through an API.
The service provisions broker and metadata-quorum workloads on Kubernetes, allocates persistent storage and network identities, exposes authenticated bootstrap endpoints, and supports topic administration, quotas, scaling, rolling version upgrades, and deletion.
Desired state must converge after controller restarts and duplicate events; an acknowledged cluster remains reachable at a stable endpoint while pods, nodes, and zones change. Customer data stays in the data plane, while the shared control plane stores lifecycle intent and status.
Design the resource model, APIs, controllers, Kubernetes and storage layout, placement, networking, security, upgrade and scale workflows, monitoring, noisy-neighbor boundaries, and failure recovery.
Kafka protocol internals, connector products, schema registries, billing, and cross-cloud federation are outside scope unless needed to defend an interface.
What to build
- A customer can idempotently create, inspect, scale, upgrade, and delete a dedicated managed Kafka cluster.
- Authorized producers and consumers can reach stable bootstrap endpoints and obtain tenant-scoped credentials without learning cluster-management internals.
- A customer can create and configure topics within validated durability, partition, retention, and throughput quotas.
- The service automatically restores desired broker capacity after pod, node, zone, or controller failure and performs observable rolling maintenance.
At scale
- For this exercise, operate 10,000 dedicated clusters across three regions, with 3 to 30 brokers per cluster and up to 100,000 partitions in the largest cluster.
- For this exercise, support 2,000 concurrent lifecycle operations and make control-plane status updates visible within 30 seconds at p95; cluster creation may take up to 15 minutes.
- For this exercise, offer a three-zone regional tier with replication factor three and target recovery from one broker or node loss without acknowledged-message loss under the configured durability policy.
- For this exercise, preserve customer endpoints and logical cluster identity across pod replacement, node draining, horizontal growth, and rolling upgrades; control-plane metadata must survive a regional control-plane outage.