Cache stampedes without the folklore

A cache miss is cheap when one client pays for it. A stampede is what happens when many clients discover the same miss at once and all race to rebuild the same value.

The failure shape

1. A hot key expires (or never existed under a new shard).
2. Latency dips briefly while the backend does useful work for the first requester.
3. Then every other waiter times out or queues — and the origin saturates.

The cache did its job for hits. The miss path had no single-flight or soft TTL, so traffic amplified instead of collapsing to one rebuild.

Patterns that actually help

Request coalescing (single-flight): the first miss starts a fill; peers wait on that fill instead of issuing parallel origin calls. Language runtimes and libraries expose this as singleflight, Group.Do, or an in-process mutex map keyed by cache key.

Soft expiry / early refresh: serve slightly stale data while one worker refreshes. Readers never see a hard miss for popular keys — they see last-known-good plus a background fill.

Probabilistic early expiry: jitter the TTL so a million keys do not expire on the same wall-clock second. Cheap, surprisingly effective at the fleet level.

Negative caching: cache "not found" briefly so a missing key does not become a permanent origin DoS.

What to measure

• Origin QPS per key on miss windows (not just aggregate hit rate).
• Fill latency p99 vs read latency p50 — stampedes show up as origin fan-out, not as "cache is empty."
• Lock / wait time inside the coalescer under load tests.

Takeaway

Caching is a control system for load, not a bag of key-value slots. Design the miss path with the same care as the hit path — that habit matters in any language, any CDN, any Redis cluster.