Resilience

A cache is an optimisation. It sits in front of things that are slower than it, and when it is working nobody thinks about it. The dangerous property is the one it acquires the moment you rely on it: everything now goes through it, including the requests it was never meant to be load bearing for.

So there is one rule underneath every mechanism on this page. A cache failure degrades the request. It does not fail it.

Fail open

An L2 read failure becomes a miss, so getOrSet can use the origin loader. A bus publish failure is reported without rejecting an already successful cache load. Errors and skipped calls remain visible through the cache event stream.

This does not suppress loader errors, origin overload, lock contention deadlines, or detected lease loss. Those can reach the caller when no eligible stale value exists.

With managed Redis Pub/Sub, a subscription gap makes local invalidation state untrusted. L1, negative entries, and stale fallbacks are flushed or bypassed until reconnect recovery restores trust. For other buses, account for their delivery and recovery behavior when choosing TTLs.

Corrupt encoded values also become misses. See read compatibility for unavailable decoders and damaged payloads.

Circuit breakers

Failing open still means trying. Trying costs a connection from the pool, a socket timeout's worth of latency, and a slot in your request path, per request, for as long as the dependency stays sick. Under load that queue is its own outage.

A circuit breaker stops the attempt. Two independent ones exist, one for L2 and one for the event bus, and both take the same shape:

interface CircuitBreakerOptions {
  enabled?: boolean
  failureThreshold?: number
  cooldownMs?: number
}

import { LazyLayersCache, RedisStore } from 'lazy-layers-cache'
import { redis } from './redis.js'
import { bus } from './bus.js'

export const cache = new LazyLayersCache({
  l2: new RedisStore(redis, { prefix: 'app:cache:' }),
  eventBus: bus,
  source: process.env.INSTANCE_ID ?? `srv-${process.pid}`,

  resilience: {
    l2CircuitBreaker: {
      // Consecutive failures before calls stop being attempted. Default 3.
      failureThreshold: 5,
      // How long calls are skipped before one probe is allowed through.
      // Default 30_000.
      cooldownMs: 10_000,
    },
    eventBusCircuitBreaker: {
      // Keep this generous and the cooldown short. See the warning below.
      failureThreshold: 10,
      cooldownMs: 2_000,
    },
  },
})

Three states, in the standard shape:

A skipped call emits l2:skipped or event-bus:publish-skipped carrying the state, so the breaker opening is something you alert on rather than infer from a latency graph. A success anywhere resets the failure count to zero, which is why the threshold counts consecutive failures and a store that fails one call in ten never trips.

Tuning is numbers. A threshold of three trips on a single blip during a Redis failover. A thirty second cooldown keeps you degraded long after the store came back. Five failures and ten seconds is a reasonable starting pair for Redis on the same network, and you should move both based on what your l2:skipped rate looks like during a real failover rather than on this paragraph.

Stale fallback

When a loader fails you often have a perfectly serviceable slightly older answer sitting in memory. Fail-safe returns it instead of propagating the error.

Values are remembered as stale at write time, kept for staleTtlMs past their normal lifetime, and returned when the loader throws. Serving one emits stale:hit with a reason of loader-error, soft-timeout, hard-timeout, or lock-timeout.

export const cache = new LazyLayersCache({
  ttlMs: 5 * 60 * 1000,

  failSafe: {
    // How long past normal expiry a value stays usable as a fallback.
    staleTtlMs: 60_000,
  },
})

staleTtlMs must be positive. Fail-safe is already enabled with a 30 second stale window, so the example only changes that window.

Negative caching

Asking repeatedly for something that does not exist runs the loader repeatedly. A crawler walking deleted IDs, or a client retrying a 404 in a tight loop, pushes full origin load through a cache that only remembers hits.

A loader that resolves to undefined records the absence. Later reads for that key return undefined without calling the loader, and emit miss with level negative. That check runs before any layer is touched, so a known miss costs nothing.

export const cache = new LazyLayersCache({
  ttlMs: 5 * 60 * 1000,

  negativeCache: {
    // Already defaults to 10 seconds. Tune for your freshness needs.
    ttlMs: 10_000,
    // Bounds the map. The oldest entry is dropped when the cap is reached.
    maxEntries: 5_000,
  },
})

The TTL is the window in which a newly created record still looks absent, so keep it short. Ten seconds absorbs a retry storm while staying well inside human patience. An invalidate call on the key clears the negative entry immediately, and so does a set event arriving from a peer, so writing through your own API stays consistent regardless of the number.

Timeouts, and the signal you have to pass on

A loader with no ceiling holds a request open for as long as the origin keeps the socket alive, which on a struggling database is minutes. Two timeouts bound that, for different reasons.

The soft timeout is a latency budget. You already hold an answer that is slightly old, so you return it rather than making the user wait for a fresher one. The hard timeout is for a loader that is not coming back at all. When the soft conditions are met the soft path takes precedence, so a request never waits hardMs when a stale answer was available at softMs.

Both fire controller.abort() on the AbortSignal handed to your loader. That signal is the only lever the cache has, because it cannot reach inside your database client and cancel a query it never issued.

import { cache } from '../cache/index.js'

export async function getUser(id) {
  return cache.getOrSet(`user:${id}`, async ({ signal } = {}) => {
    // Passing the signal is what makes the timeout cancel work rather than
    // just stop waiting for it.
    const res = await fetch(`${process.env.USER_API}/users/${id}`, { signal })
    return res.ok ? res.json() : undefined
  })
}

What is already on

Every protection is opt-out. A cache that only defends you once you have read the whole options reference defends nobody on their first deploy, which is exactly when it matters.

Tuning for production means changing these numbers, not setting anything to false.

Where to next