Preparing your design workspace

Loading the problem and your latest saved attempt…

Design an Online SQL Service Over a Data Lake

Run isolated interactive SQL against versioned table snapshots with scalable planning, execution, and result delivery.

Problem statement

The source asks for an online SQL service over Apache Iceberg data. For this exercise, assume authenticated tenants submit read-only SQL against cataloged tables, pin one consistent table snapshot per query, inspect status, cancel work, and page through results.

The platform plans distributed scans, enforces tenant and query resource limits, handles schema and partition evolution, and stores bounded result sets for later retrieval. Table writes, ETL authoring, and the implementation of the underlying object store are outside scope.

What to build

  1. Tenants can submit read-only SQL with an explicit catalog and receive a stable query identity.
  2. Tenants can inspect queued and running state, cancel a query, and distinguish rejected, failed, cancelled, and completed outcomes.
  3. Each query plans and executes against one consistent set of Iceberg table snapshots despite concurrent catalog updates.
  4. Authorized clients can page through a bounded immutable result set and its schema until retention expires.

At scale

  • For this exercise, assume 5,000 concurrent queries across 1,000 tenants and 10 PB of Iceberg table data.
  • For this exercise, selective interactive queries should return their first result page within 10 seconds at p95, while large scans may run for 30 minutes.
  • For this exercise, catalog snapshot changes should be visible to newly submitted queries within 60 seconds without changing already admitted queries.
  • For this exercise, retain completed result sets for 24 hours and enforce per-tenant CPU, scan-byte, concurrency, and result-size limits.