Batches, namespaces and freshness
Load many keys with one call, group keys you can clear together, refresh values before they expire, serve stale values while refreshing, and spread expiry.
In lazy-layers-cache, staleWhileRevalidate: true makes getOrSet return a recently expired value at once and reload the key once in the background, and earlyRefresh: 0.8 reloads a popular key before it expires, so readers do not wait on the origin. getOrSetMany loads every miss in a batch with one loader call, and namespace() groups keys you can clear together.
Batch reads and loads
const [a, b, c] = await cache.getMany(['user:1', 'user:2', 'user:3'])
const users = await cache.getOrSetMany(['user:1', 'user:2', 'user:3'], async (missing) => {
// One query for every miss. Return a Map, or an array in the same order.
const rows = await db.users.findMany({ where: { id: { in: missing.map(idOf) } } })
return new Map(rows.map((row) => [`user:${row.id}`, row]))
})getOrSetMany calls the loader once with the distinct keys that missed. A key the loader leaves out is a miss and is remembered by the negative cache. Keys already being loaded by another call are joined, and concurrent getOrSet calls for keys in the batch join it. Like a single load, a batch never overwrites a write or delete that happened while it was loading.
A batch takes one origin slot and no cross-process lease, so two instances can load the same missing key at the same moment.
setMany(entries) and deleteMany(keys) write and delete with the same guarantees as their single-key versions.
Namespaces
const users = cache.namespace('users')
const tenant = cache.namespace('tenant:42')
await users.set('7', { name: 'Ada' }) // stored as users:7
await tenant.namespace('orders').get('9') // tenant:42:orders:9
await tenant.clear() // invalidates tenant:42:* here and on peersA namespace shares its parent's layers, memory budget, connections and event bus. Only key names differ. clear() is a pattern invalidation of the namespace's prefix, so it reaches L2 and every peer. A name cannot contain *.
Early refresh
const cache = createCache({ ttlMs: ms('10m'), earlyRefresh: 0.8 })A popular key should never expire under load. With earlyRefresh: 0.8, the first getOrSet hit after 80% of a value's lifetime returns the cached value and reloads the key once in the background. The new value replaces the old one when it lands, so callers never wait on the origin for a key that keeps being read.
- The lifetime is the L2 TTL when there is an L2, and the L1 TTL otherwise. A value read back from L2 keeps its age: one written 9 minutes ago is refreshed on its first hit, not 8 minutes later.
- The reload is the key's in-flight load. A burst of hits starts one reload, callers that miss L1 meanwhile join it, and it goes through the origin gate and the distributed lock. When another instance holds the lock, this one waits for its result instead of loading the key again.
- A reload that fails, or finds nothing, keeps the current value and is retried halfway to its expiry, not on every hit.
- A delete or invalidation during the reload wins: the reloaded value is discarded.
- It applies to
getOrSetwith the built-in L1.getnever reloads, because it has no loader.
Reloads started are counted in lazycache_early_refresh_total and reported as refresh:early events.
Early refresh keeps popular keys from expiring; stale-while-revalidate, below, covers keys that did expire.
Stale-while-revalidate
const cache = createCache({
staleWhileRevalidate: true,
failSafe: { staleTtlMs: 60_000 },
})When a key has expired but a recent value is still held, getOrSet returns that value immediately and refreshes the key once in the background. Callers never wait on the origin while a usable value exists, and a burst of callers starts one refresh. The window is failSafe.staleTtlMs past the value's expiry. A key that was deleted or invalidated is never served this way.
Stale values served are counted in lazycache_stale_served_total{reason="revalidate"}.
Spreading expiry
const cache = createCache({ ttlMs: ms('10m'), ttlJitter: 0.1 })Keys written together (after a deploy, or by a batch job) otherwise expire together and miss together. ttlJitter: 0.1 scales each write's TTL by a random factor between 0.9 and 1.1, the same factor for L1 and L2, so expiry is spread out.
Readable durations
Every option ending in Ms takes milliseconds. ms() converts readable durations once, where the option is written:
import { ms } from 'lazy-layers-cache'
createCache({ ttlMs: ms('2.5h'), failSafe: { staleTtlMs: ms('30s') } })Units are ms, s, m, h and d.
Questions
What is the difference between earlyRefresh and staleWhileRevalidate?
earlyRefresh acts before expiry: the first hit after the given share of the lifetime returns the cached value and starts one background reload. staleWhileRevalidate acts after expiry: within failSafe.staleTtlMs, an expired value is returned at once while one refresh runs.
How do I stop many keys from expiring at the same moment?
Set ttlJitter, for example 0.1, which scales each write's TTL by a random factor between 0.9 and 1.1, the same factor for L1 and L2.
Does getOrSetMany coordinate loads across instances?
No. A batch takes one origin slot and no cross-process lease, so two instances can load the same missing key at the same moment. Within one process, keys already in flight are joined.