Subscribe

System design basics: latency, throughput and back-of-the-envelope estimates

Latency numbers I measured on my own Mac, QPS and storage maths, read/write ratios and Little's law, worked end to end for shorty, a URL shortener in Go with Postgres and Redis.

System design basics: latency, throughput and back-of-the-envelope estimates

Every system design discussion I have been in sooner or later hits the same question: how big is this thing, really? Most of the bad designs I have seen were not wrong in the boxes and arrows. They were wrong by a factor of 100 in the numbers. In this blog we will look at the few numbers worth knowing, how to turn a product idea into requests per second and gigabytes, and then we will work it all out for shorty, the URL shortener this series builds.

Level: beginner. You need nothing installed to follow the maths. If you want to run my measurements yourself you need Go (I used 1.27.3), and Docker for Postgres 18.7 and Redis 8.10. The design of shorty itself is in Designing a URL shortener from requirements to scale, this post is about the numbers behind it.

Requirements

An estimate is only as good as the questions you asked first. For shorty I wrote these down:

  • Users create short links with POST /links and get back something like https://sho.rt/Ab3dE9x.
  • Anyone who opens a short link gets a 302 redirect from GET /{code}.
  • Links don't change once created. They can be deleted.
  • 10 million new links a month.
  • Every link is opened about 100 times on average, so reads are 100 times the writes.
  • Keep links for five years.
  • A redirect should feel instant. I'll say p99 under 50 ms at the server.

The last two are the ones people forget. Retention decides storage, and the latency target decides whether you need a cache at all.

Latency numbers I actually measured

Everybody copies the same "latency numbers every programmer should know" table. It is a good table, but it is old, and I wanted numbers from my own machine. So I wrote a small Go program that times a few operations, with Postgres and Redis running in Docker on the same Mac (an Apple M5), and shorty itself running natively.

func measure(name string, n int, f func()) {
    d := make([]time.Duration, n)
    for i := range d {
        start := time.Now()
        f()
        d[i] = time.Since(start)
    }
    slices.Sort(d)
    fmt.Printf("%-40s p50 %10v   p99 %10v\n", name, d[n/2], d[n*99/100])
}

// tiny is for operations faster than the clock: time.Now() itself costs
// tens of nanoseconds, so we let testing.Benchmark run them in a loop.
func tiny(name string, f func()) {
    r := testing.Benchmark(func(b *testing.B) {
        for b.Loop() {
            f()
        }
    })
    fmt.Printf("%-40s     %10v per op\n", name, time.Duration(r.NsPerOp()))
}

The tiny helper is there because of my first run. I timed a map lookup with time.Now() around it and got 41 ns. That was the cost of reading the clock, the map lookup was hiding inside it. With testing.Benchmark and b.Loop it came out at 6 ns.

DATABASE_URL='postgres://postgres:pw@127.0.0.1:55439/shorty?sslmode=disable' \
REDIS_ADDR=127.0.0.1:56389 \
SHORTY_URL=http://localhost:18080/docs \
go run ./latency
map lookup (100k keys)                              6ns per op
sha256 of 1 KB                                    385ns per op
json.Marshal small map                            442ns per op
write 4 KB + fsync (local SSD)           p50 4.017458ms   p99 7.114875ms
redis GET (localhost, docker)            p50   74.917µs   p99  238.667µs
postgres SELECT by code (localhost)      p50   83.541µs   p99  119.416µs
shorty redirect, full HTTP (localhost)   p50  127.667µs   p99  299.167µs

A few things jumped out. Anything in memory is nanoseconds. Anything that crosses a socket, even on localhost, is around 100 microseconds, so roughly ten thousand times slower than a map lookup. And the fsync was the slowest line by far at 4 ms. On macOS Go's Sync asks for a full flush to the disk, which is honest but slow. That's why databases batch their commits.

Redis and Postgres came out almost the same here. That surprised me a bit, but it makes sense: the row is in Postgres's memory and the query is a primary key lookup, so both are mostly network round trip. In AWS, add 0.3 to 1 ms for a hop inside one availability zone, and more across zones. The rule of thumb I keep is simple. Memory is ns, same machine is ~100 µs, same data centre is ~1 ms, another region is 50 to 150 ms.

Estimates for shorty

Now the arithmetic. There are 86,400 seconds in a day, I round it to 100,000 when doing it in my head and correct later.

writes   10,000,000 links / month
         = 10,000,000 / (30 * 86,400)          ~ 4 per second average
         peak (assume 5x average)              ~ 20 per second

reads    100 reads per link -> 1,000,000,000 / month
         = 1,000,000,000 / (30 * 86,400)       ~ 390 per second average
         peak (5x)                             ~ 2,000 per second

storage  rows in 5 years = 10M * 12 * 5        = 600,000,000
         bytes per row (measured, see below)   ~ 190 with 68-char URLs
         assume longer real URLs               ~ 240 bytes per row
         total                                 ~ 144 GB (table + index)

codes    62^7 = 3.5 trillion possible codes
         600M used = 0.017% full

The peak factor of 5 is a guess. For a link shortener it can be much worse, one link in a viral post gets most of the traffic for an hour. That one is a caching problem more than a capacity problem, and caching gets its own post in this series.

Checking the row size

I did not want to guess bytes per row, so I loaded a million rows into shorty's real links table on Postgres 18.7 and asked Postgres.

INSERT INTO links (code, url)
SELECT substr(md5(g::text), 1, 7),
       'https://example.com/articles/' || md5(random()::text) || '/' || g
FROM generate_series(1, 1000000) g
ON CONFLICT DO NOTHING;

VACUUM ANALYZE links;

SELECT count(*),
       pg_size_pretty(pg_table_size('links'))   AS tbl,
       pg_size_pretty(pg_indexes_size('links')) AS idx,
       pg_total_relation_size('links') / count(*) AS bytes_per_row,
       avg(length(url))::int AS avg_url
FROM links;
 count  |  tbl   |  idx  | bytes_per_row | avg_url
--------+--------+-------+---------------+---------
 998155 | 128 MB | 51 MB |           187 |      68

So 187 bytes for a 68 character URL, including the primary key index and the unique index on code. Fixed overhead per row is bigger then people expect.

And look at the count. The INSERT reported 998,154 rows, not a million (the extra row in the count is a link I had made by hand earlier). ON CONFLICT dropped 1,846 rows because 7 hex characters only give 16^7 = 268 million codes, and the birthday maths says a million random picks from that will collide about n²/2N = 1,863 times. I didn't plan that, but its a nice check that the formula works. shorty uses 7 base62 characters instead, 3.5 trillion codes, so at 600 million links a new code collides about once in 6,000 tries. Three attempts is plenty.

Throughput, latency and Little's law

Requests per second alone doesn't tell you how many servers or connections you need. Little's law does:

requests in flight = throughput * latency

I wrote a small load generator that runs N workers against one URL for a few seconds and ran it against shorty on the same Mac, redirect for a link already in Redis.

go run ./load -url http://localhost:18080/docs -c 50 -d 5s
c=1    1094 req/s   p50 333µs   p99 9.1ms
c=10   4824 req/s   p50 895µs   p99 17.3ms
c=50  10621 req/s   p50 1.5ms   p99 201.5ms

Two lessons from that run. More concurrency gave more throughput but latency went up with it, and at 50 workers the p99 jumped to 200 ms while the p50 stayed under 2 ms. The machine was saturated. The load generator, shorty, Docker and the JSON request log were all fighting for the same cores, so don't read these as production numbers. Read the shape.

For shorty at the 2,000 req/s peak with, say, 5 ms average latency in AWS, that is 2,000 * 0.005 = 10 requests in flight. Tiny. One small instance can do it and I'd run two only so a deploy or a crash doesn't take redirects down.

The database pool works the same way. If 10% of redirects miss the cache, that's 200 queries a second, and at 1 ms each Postgres has 0.2 connections busy on average. shorty's default DB_MAX_CONNS=10 is far more than enough. I like this check because it stops people sizing pools at 100 "to be safe".

High-level picture

            ~2,000 req/s peak (reads)        ~20 req/s peak (writes)
                     |                                 |
                     v                                 v
              +-------------+                   +-------------+
              | load        |------------------>| shorty x 2  |
              | balancer    |                   | (Go, :8080) |
              +-------------+                   +------+------+
                                                       |
                                  hit ~90%  +----------+----------+  miss ~10%
                                            v                     v
                                     +-------------+       +-------------+
                                     | Redis       |       | Postgres    |
                                     | hot links   |       | 144 GB in   |
                                     | ~200 MB     |       | 5 years     |
                                     +-------------+       +-------------+

The Redis size is another estimate: if a million links are hot at any time and each key plus URL plus overhead is around 200 bytes, that is 200 MB. A small instance.

Trade-offs and where estimates lie

Averages hide peaks, so always multiply. I used 5x, for anything with marketing pushes or viral content I'd use 10x and check again with real traffic once it exists.

Storage estimates are usually on the low side. Backups, WAL, replicas and indexes you add later all multiply the 144 GB. I'd budget three times the raw number.

And the read/write ratio decides the design more than the totals do. At 100:1 shorty is a read-heavy system, so the cache and the redirect path get all the attention, and the write path can stay boring. If it was 1:1 I would be thinking about the database first.

The code for both programs (latency and load) is small enough to copy into any project. Run them on your own machine before you trust anybody's table, including mine.