Handle failure

A cache sits between your users and everything slow. That position is also what makes it dangerous: a cache that throws when its store is sick has turned an optimisation into an outage. This page is about what your users see when something underneath stops answering.

Fail open is the stance

L2 read failures become cache misses, allowing getOrSet to reach the loader. An event-bus publish failure is reported without rejecting a successful cache load. Circuit breakers and bounded queues limit repeated work against unavailable dependencies.

Fail-open does not guarantee a successful request. Loader failures, origin overload, contention deadlines, and detected lease loss can still reach the caller when no eligible stale value exists. Stampede protection explains the distinction between a Redis transport failure and a contended or lost lease.

Circuit breakers

Failing open still means trying, and trying costs a connection, a socket timeout, and a slot in your request path. A circuit breaker stops you queueing work against a dependency that is already struggling.

Two independent breakers exist, one for L2 and one for the event bus. Both take the same CircuitBreakerOptions, and both are constructed for every cache you build, so their defaults are already live: three consecutive failures to open, then a thirty second cooldown. Configuring them is how you replace those numbers with numbers that fit your dependency.

import { LazyLayersCache, RedisStore } from 'lazy-layers-cache'
import { redis } from './redis.js'
import { bus } from './bus.js'

export const cache = new LazyLayersCache({
  l2: new RedisStore(redis, { prefix: 'app:cache:' }),
  eventBus: bus,
  source: process.env.INSTANCE_ID ?? `srv-${process.pid}`,

  resilience: {
    l2CircuitBreaker: {
      // Consecutive failures before the breaker opens. Default 3.
      failureThreshold: 5,
      // How long calls are skipped before one probe is allowed through.
      cooldownMs: 10_000,
    },
    eventBusCircuitBreaker: {
      failureThreshold: 3,
      cooldownMs: 5_000,
    },
  },
})

The breaker has three states, which is the standard shape.

A skipped call emits l2:skipped or event-bus:publish-skipped carrying the state, so the breaker opening is something you can alert on rather than infer from a latency graph.

Tune with numbers. failureThreshold defaults to 3 and cooldownMs to 30,000. A low threshold trips on a single blip, and a long cooldown keeps you degraded after the store has recovered. Five failures and ten seconds is a reasonable starting pair for Redis on the same network.

Stale fallback

When a loader fails, you often have a perfectly serviceable slightly older answer sitting in memory. Stale fallback returns it instead of propagating the error.

It is on by default with a 30,000 ms window. Set staleTtlMs when you need a different recovery window.

export const cache = new LazyLayersCache({
  ttlMs: 5 * 60 * 1000,

  failSafe: {
    // How long past normal expiry a value stays usable as a fallback.
    staleTtlMs: 60_000,
  },
})

Values are remembered as stale when they are written, and kept for staleTtlMs beyond their normal lifetime. If the loader then throws, the stale value is returned and stale:hit is emitted with a reason of loader-error, soft-timeout, or hard-timeout.

import { cache } from './index.js'

cache.on((event) => {
  if (event.type === 'stale:hit') {
    logger.warn({ key: event.key, reason: event.reason }, 'served stale')
  }
})

Stale fallback is also what makes a soft timeout meaningful, which is the next section.

L1 pressure does not turn an L2 hit into a miss

The built-in L1 shares a byte budget with its stale snapshots. During memory pressure it evicts LRU entries and can reject a new L1 promotion. An L2 hit still returns to the caller. It simply does not consume more local heap, and emits promotion:bypassed with pressure or budget.

Peer priming follows the same rule. Under pressure, a received value removes an obsolete L1 entry before the replacement promotion is bypassed. The server then reads the shared value from L2 on demand instead of knowingly serving the old L1 copy.

Treat sustained promotion:bypassed events as a capacity signal. Check the instance memory limit, payload sizes, admission.maxEntryBytes, and L2 latency before increasing maxMemory. The budget is not a process RSS guarantee, so an oversized L1 ceiling can still leave too little heap for the application.

Loader timeouts

A loader with no ceiling can hold a request open for as long as the origin is willing to keep the socket alive, which on a struggling database can be minutes. Two timeouts bound that, and they exist for different reasons.

export const cache = new LazyLayersCache({
  ttlMs: 5 * 60 * 1000,

  failSafe: { staleTtlMs: 60_000 },

  timeouts: {
    // Latency budget. Needs fail-safe on and a stale value present.
    softMs: 200,
    // Absolute ceiling. Always applies.
    hardMs: 2_000,
  },
})

The soft timeout is a latency budget. You already hold an answer that is slightly old, so you return it rather than making the user wait for a fresher one. The hard timeout is for a loader that is not coming back. When the soft path applies it takes precedence, so a request never waits hardMs when a stale answer was available at softMs.

Pass the signal to your I/O

Both timeouts abort the loader's AbortSignal. That is the only lever the cache has, because it cannot reach inside your database client and cancel a query it never issued.

import { cache } from '../cache/index.js'

export async function getUser(id) {
  return cache.getOrSet(`user:${id}`, async ({ signal } = {}) => {
    // Passing the signal is what makes the timeout cancel work rather than
    // just stop waiting for it.
    const res = await fetch(`${process.env.USER_API}/users/${id}`, { signal })
    return res.ok ? res.json() : undefined
  })
}

Negative caching

Asking repeatedly for something that does not exist means running the loader repeatedly. A crawler walking deleted IDs, or a client retrying a 404, will happily push full origin load through a cache that only remembers hits.

Negative caching records the absence. A loader that resolves to undefined writes a short lived negative entry, and later reads for that key return undefined without calling the loader, emitting miss with level negative.

export const cache = new LazyLayersCache({
  ttlMs: 5 * 60 * 1000,

  negativeCache: {
    // Keep it short. This is the window in which a newly created record
    // still looks absent.
    ttlMs: 10_000,
    maxEntries: 5_000,
  },
})

A positive ttlMs is required. Without one, nothing is recorded. maxEntries bounds the map, and the oldest entry is evicted when the cap is reached.

Ten seconds is usually enough to absorb a retry storm while staying short enough that a just created record becomes visible quickly. A delete on the key clears the negative entry immediately, so writing through your own API stays consistent regardless of the TTL.

Watching failures

Every failure path emits an event. A reasonable production subscription:

import { cache } from './index.js'

cache.on((event) => {
  switch (event.type) {
    case 'l2:error':
      // Carries the breaker state, so you can alert on the transition to open.
      metrics.increment('cache.l2.error', { operation: event.operation, state: event.state })
      break
    case 'l2:skipped':
      metrics.increment('cache.l2.skipped', { state: event.state })
      break
    case 'stale:hit':
      metrics.increment('cache.stale', { reason: event.reason })
      break
    case 'loader:timeout':
      metrics.increment('cache.loader.timeout')
      break
    case 'event-bus:publish-error':
      metrics.increment('cache.bus.error')
      break
    case 'event-bus:publish-skipped':
      // Remember: these events are dropped, not queued for retry.
      metrics.increment('cache.bus.skipped')
      break
  }
})

Where to next