Design Packet Routing Between a Hub and Node Servers
Design a central hub and its protocol that forward packets between node servers quickly, keep each sender's packets in order, keep memory bounded when a node falls behind, and recover when the hub fails.
Problem statement
For this exercise, assume a small cluster in one data center: one hub server and up to 64 node servers. Node servers never talk to each other directly. Each node keeps one long-lived TCP connection to the hub, and every packet goes from the sending node to the hub, which forwards it to the destination node.
A packet is a binary message whose payload the hub does not read.
Design the hub and the protocol between the hub and the nodes. A node must be able to join, send a packet to one other node by its node ID, send a packet to every other node, and leave. The hub must forward packets quickly and tell a sender when its packet cannot be delivered.
For this exercise, assume the hub keeps packets only in memory; nothing is written to disk. Packets from one sender to one destination must arrive in the order they were sent, with no duplicates, while both nodes and the hub stay up. There is no ordering promise across different senders.
When a node or the hub restarts, packets in flight may be lost, but the receiving node must be able to tell that something may be missing. Assume the hub has a configured list of allowed node IDs.
A standby hub machine is available; after a hub failure, traffic may pause for a few seconds, and nodes must reconnect without an operator. Encryption, node authentication beyond the allowed-ID list and routing across data centers are out of scope.
What to build
- A node can join with its node ID and become reachable, and the hub stops routing to it when it leaves or stops responding.
- A node can send a packet to one other node by node ID, and the destination receives that sender's packets in the order they were sent.
- A node can send one packet that the hub delivers to every other connected node.
- A sender is told when the hub cannot deliver its packet, for example because the destination is unknown, gone or too far behind.
At scale
- For this exercise, assume up to 64 node servers and a peak of 1 million packets per second through the hub in total, with any single node sending up to 200,000 packets per second.
- For this exercise, assume most packets are about 200 bytes and none is larger than 64 KiB.
- For this exercise, assume the hub should add less than 100 microseconds at p99 under normal load, and should notice a dead node within 1 second.
- For this exercise, assume the hub may hold at most 4 MiB of packets waiting for any one node; a node that falls further behind must not slow delivery to the others.