Estimation that holds up under questioning

Key points

• Estimation is not about being right. It is about being within an order of magnitude and showing the reasoning, so the design can be judged against a number rather than a vibe.
• Work from users to requests to bytes. Daily active users, actions each, a peak-to-average ratio of roughly 2 to 5, then bytes per action - each step visible and challengeable.
• Separate storage from throughput. They lead to different designs: a system that stores petabytes but serves 100 requests per second is nothing like one storing gigabytes at 100,000 requests per second.
• Read-to-write ratio decides the architecture more than raw volume. 1000:1 says caching and read replicas; 1:1 says the write path is the hard part.
• Keep a handful of latency numbers in mind: memory reference ~100ns, SSD read ~100us, network round trip within a datacentre ~500us, and cross-continent ~150ms - dominated by the speed of light and not improvable.
• State the assumption out loud whenever you use one. An estimate with visible assumptions can be corrected; a number without them can only be believed or doubted.

Example

/*
 * WORKED EXAMPLE - a photo sharing service.
 *
 * ASSUMPTIONS (state them, so they can be challenged):
 *   100M daily active users
 *   each uploads 0.1 photos/day, views 50
 *   average photo 2 MB, thumbnail 20 KB
 *   peak = 3x average
 *
 * WRITES:
 *   100M x 0.1              = 10M uploads/day
 *   10M / 86,400            = ~115 uploads/sec average
 *   x3 peak                 = ~350/sec peak
 *   115/s x 2 MB            = ~230 MB/s ingest
 *
 * READS:
 *   100M x 50               = 5B views/day
 *   5B / 86,400             = ~58,000/sec average
 *   x3 peak                 = ~175,000/sec peak
 *   -> read:write is 500:1. THE CACHE IS THE SYSTEM.
 *
 * STORAGE:
 *   10M/day x 2 MB          = 20 TB/day
 *   x 365                   = ~7 PB/year
 *   x3 replication          = ~21 PB/year
 *   -> object storage, not a database. Metadata only in the DB.
 *
 * BANDWIDTH OUT:
 *   58,000/s x 20 KB thumb  = ~1.2 GB/s   -> a CDN is mandatory,
 *                                            not an optimisation
 *
 * WHAT THE NUMBERS DECIDED:
 *   500:1 read ratio  -> CDN + aggressive caching
 *   7 PB/year         -> blob store, DB holds metadata only
 *   350 writes/sec    -> modest; the write path is NOT the hard part
 */

/*
 * NUMBERS WORTH KNOWING:
 *
 *   L1 cache reference                 1 ns
 *   main memory reference            100 ns
 *   SSD random read              100,000 ns   (100 us)
 *   round trip in a datacentre   500,000 ns   (0.5 ms)
 *   disk seek (spinning)      10,000,000 ns   (10 ms)
 *   round trip CA to Europe  150,000,000 ns   (150 ms)  <- physics
 *
 *   1 day        = 86,400 s   (~100k, for mental arithmetic)
 *   1M/day       = ~12/sec
 *   1B/day       = ~12,000/sec
 */

Work users to requests to bytes with the assumptions visible - and let the read-to-write ratio choose the architecture.