TTL & eviction
Cache memory is finite and data goes stale. What do you keep, and for how long?
Open the interactive version → diagrams, practice & moreThe problem
Cache memory is finite and data goes stale. What do you keep, and for how long?
The idea
TTL expires entries after a time; eviction policies (LRU, LFU) drop entries when memory is full.
How it works
TTL bounds staleness by expiring entries after a time; eviction frees memory when the cache is full. LRU is a fine default but a one-time scan (a batch job touching cold keys) evicts your hot working set — it isn't scan-resistant. LFU keeps frequently-used keys but adapts slowly to shifting popularity. Modern caches use W-TinyLFU / ARC, which approximate "frequently AND recently used" and resist scans — why Caffeine and others adopted them.
The tradeoff
Short TTL = fresher but more DB load; long TTL = faster but staler. Eviction policy matters most when the working set doesn't fit in memory: pure LRU is simple but scan-vulnerable; LFU resists scans but can pin stale-popular keys; W-TinyLFU balances both at a little bookkeeping cost. If the working set fits in RAM, policy barely matters — size up instead.
In the wild
A user profile cached for 60s; a product price for 5s; static assets for a year.
Interview deep dive
Flow
- Estimate the hot working-set size vs available cache memory.
- Set TTL from how stale each key may safely be.
- Choose LRU (simple), LFU, or W-TinyLFU (scan-resistant).
- Watch hit ratio and eviction rate; resize if churn is high.
Watch for
- A cold scan evicts the hot set under pure LRU.
- TTL (staleness) and eviction (memory) are separate controls.
- Big values skew per-key memory accounting and eviction.
Interviewer trap
Distinguish TTL from eviction, and flag pure-LRU's scan vulnerability.