Preparing your design workspace

Loading the problem and your latest saved attempt…

Low-Cost Access to a File Archive

Improve lookup and download startup for 90,000 hosted files on one existing server, while keeping stale metadata from returning the wrong file revision.

Problem statement

For this exercise, assume an archive already hosts 90,000 files of different sizes on one server. Users experience slow access when locating a file and starting a download. The budget does not permit another host.

Design a small service that supports case-sensitive filename or filename-prefix lookup and streams a selected file revision to the user.

Separate time spent finding and opening the right file from time spent transferring all its bytes. Improve lookup and time to first byte within the stated workload; do not promise that a large file finishes downloading within the lookup or first-byte latency target.

Keep request memory bounded and avoid scanning or reading every file's contents on each lookup.

For the practice lifecycle, assume trusted administrators occasionally publish, replace or remove files through a controlled local workflow. Each published revision has an immutable identity, and normal catalog changes should become visible within five minutes.

A result may be stale during that window, but a download must return the specifically selected revision or an explicit unavailable result; it must never silently return another revision because a filename or path was reused.

The host limits, APIs, lifecycle and latency targets below are authored practice assumptions. Indexing is a useful possibility, not a prescribed database or a claim about a recovered employer architecture. Global sharing, multi-device synchronization, content ranking and new storage hosts are outside this exercise.

What to build

  1. Find files by case-sensitive exact filename or prefix and return deterministically ordered metadata in bounded pages, without reading all file contents for each lookup.
  2. Stream a selected immutable file revision with bounded memory; return that revision or an explicit unavailable result rather than silently substituting newer bytes.
  3. Reflect trusted administrative publications, replacements and removals in the lookup catalog within five minutes during normal operation.
  4. Use the existing host, keep memory and background work bounded, and recover a stale or interrupted catalog update without requiring a full content scan on every request.

At scale

  • The archive contains 90,000 files. Illustrative practice limits: file sizes from 1 KiB to 2 GiB on an existing 2-vCPU, 8-GiB-RAM host whose local disk has adequate space for the archive and a small metadata index. No additional host is allowed.
  • Normal peak envelope: 30 metadata lookups per second, up to 2 download starts per second, at most 8 concurrent downloads and aggregate transferred payload at most 20 MiB/s. Return at most 100 metadata results per page; reject or queue work beyond the chosen bounded limits.
  • Within that normal envelope and available host/disk capacity, target p95 lookup latency at most 100 ms and p95 time to first byte at most 250 ms. Full download completion depends on file size and available throughput; neither target is a whole-file completion deadline.
  • A controlled administrative change should reach the active catalog within 5 minutes during normal operation. Interrupted indexing may make a catalog unavailable or stale, but no stale entry or cache may return bytes from a different revision.