L1/L2 cache system design

A hybrid L1/L2 cache system puts the same value in two places for different reasons: L1 answers fast local reads, while Redis L2 answers shared reads across a fleet. Pick one cache layer and you have picked a failure mode. The two layers exist because "fast" and "shared" are different questions, and answering them with one piece of infrastructure means answering one of them badly.

One layer, two ways to be unhappy

Memory only. Reads are as fast as a map lookup, and that is the end of the good news. Every server holds its own copy of everything, so a fleet of forty servers holds forty copies and pays for forty cold starts. A restart throws the whole thing away. Each server only ever benefits from its own traffic, so your hit rate falls as you add capacity, which is exactly backwards.

Shared store only. Now there is one copy, it survives restarts, and every server sees the same value. But the key you read four hundred times a second costs a network round trip and a decode every single time, and your read path now fails when Redis does.

Two layers is not a compromise between those. L1 answers "how fast is a repeat read on this server". L2 answers "what does the fleet agree on, and what survives a deploy". Neither answer degrades the other.

L1, the copy in this process

L1 is an LRU inside your process heap, holding Lazy Layers serializer buffers. A hit costs a map lookup plus a decode, but nothing crosses a socket. Encoding makes retained payload bytes measurable and prevents caller mutation from changing cached state.

It is created for you when you do not pass l1.

Eviction order remains LRU. The bounded history influences admission only; it does not turn the cache into LFU or add a configurable replacement policy.

L1 lives in the process, so it is per server. Three servers means three L1 caches that know nothing about each other. Closing that gap is the job of the event bus, through the lazy fan-out and invalidation, not of L1 itself.

Its blast radius is small, and that is its other virtue. A wrong value in L1 affects one server, expires at its TTL, and dies with the process.

L2, the copy everybody shares

L2 is optional and shared. RedisStore is the implementation in the package, and it stores bytes rather than objects, so values are encoded on the way in and decoded on the way out. Serialization covers the format.

import { LazyLayersCache, RedisStore } from 'lazy-layers-cache'
import { redis } from './redis.js'

export const cache = new LazyLayersCache({
  levels: {
    // Short in memory, so a wrong value cannot linger on one server.
    L1: { maxEntries: 10_000, ttlMs: 60 * 1000 },
    // Long in the shared tier, because it is what absorbs the load.
    L2: { ttlMs: 60 * 60 * 1000 },
  },

  l2: new RedisStore(redis, {
    prefix: 'app:cache:',
    deleteStrategy: 'unlink',
    scanCount: 500,
  }),
})

L2 survives a restart, is visible to every server, and catches what L1 evicts. It costs a round trip, and it can fail. When it does, the failure is treated as a miss rather than an exception, so the read falls through to your loader and the request still completes. Resilience covers what that degradation looks like.

Its blast radius is the opposite of L1's. A wrong value in L2 is served to every server and outlives every restart until something deletes it. That is why writes invalidate L2 explicitly instead of waiting for its TTL to come around.

How a read walks the layers

The promotion step normally makes the second read cheap. A value found in L2 is written into L1 before it is returned when the local budget permits. During pressure it is returned without promotion, so Redis remains available without immediately refilling the hot set.

What each layer is for

Sizing L1 by bytes and entry count

maxMemory is the hard process-local accounting ceiling and defaults to 20% of effective host or container memory. maxEntries remains a second guard against a huge population of tiny keys.

Budget encoded payloads, estimated key and metadata overhead, and fail-safe snapshots. Returned decoded objects and values still owned by application code are outside this ledger, so the ceiling is not a process RSS guarantee.

ttlMs is a size dial too, and an easier one to reason about. A shorter L1 TTL means fewer entries are alive at any moment, which caps memory by turnover rather than by count.

Under Node clustering, every worker keeps its own L1 and process-local budget. Divide a host allowance across workers or set a container limit for each worker.

TTLs resolve from most specific to least

1. per call options.levels.L1.ttlMs or options.levels.L2.ttlMs
2. per call options.ttlMs
3. constructor levels.L1.ttlMs or levels.L2.ttlMs
4. constructor ttlMs
5. DEFAULT_CACHE_TTL_MS, which is one hour

A short L1 TTL with a longer L2 TTL is the usual arrangement. Memory turns over quickly, the shared tier keeps absorbing the load, and no single server holds anything for long.

Running with one layer

Either layer can be removed by passing false.

L1 only is the right shape for a single process with no peers, where there is nobody to share with and nothing to invalidate. l1: false is occasionally useful when many processes share one machine and holding the same objects several times over is the thing you are trying to avoid, at the cost of a round trip on every read.

Where to next