Design Telemetry Name Reconciliation
Design a deterministic control plane and data path that converge legacy telemetry-name variants onto canonical metrics without fuzzy matching.
Problem statement
For this exercise, assume many deployed clients emit timestamped numeric telemetry under legacy spelling, casing, delimiter, or namespace variants.
Operators maintain explicit, versioned mappings from each tenant-scoped source name to one canonical telemetry identifier; fuzzy matching, clustering, and machine-learned guesses are forbidden.
New samples should be mapped before they become queryable, while already stored history must converge without hiding gaps, double counting, or silently merging unrelated metrics.
Operators need to preview mapping impact, reject collisions, publish or roll back a mapping version, monitor unmapped names, and inspect lineage from a canonical result back to its original name and rule version.
During a historical rewrite, queries must return one documented view rather than mixing rewritten and unreconciled data ambiguously. Alerting, arbitrary metric calculations, and a general-purpose time-series database are outside scope.
What to build
- Producers can idempotently submit telemetry samples and receive a canonical, unmapped, invalid, or retryable outcome without any probabilistic name inference.
- Authorized operators can preview, validate, publish, supersede, roll back, and audit explicit source-name-to-canonical-name mappings.
- Authorized users can query a canonical telemetry identifier across old and new names with freshness, reconciliation progress, and source lineage.
- Operators can run and observe retry-safe historical reconciliation without double counting samples or exposing an ambiguous mixed view.
At scale
- For this exercise, assume 100,000 active producers emit one million samples per second across 20 million source names and five million canonical telemetry identifiers.
- For this exercise, durably acknowledge 99% of valid mapped samples within one second and make them queryable within 15 seconds; unmapped names must remain durable and visible to operators.
- For this exercise, publish a validated mapping version within 30 seconds and require every ingestion decision to retain the exact mapping version used.
- For this exercise, retain original-name lineage for thirteen months and reconcile a 30-day raw-history window while live ingestion and canonical queries remain available.