Observe the cache

A cache you cannot measure is a cache you cannot tune. Worse, it is a cache you cannot debug, because every symptom looks the same from outside: the response was slow, or the response was wrong, and nothing in your application code says which layer decided that.

Three ways to see inside, from cheapest to most interactive.

The event stream

Everything the cache does emits an event, and cache.on() is how you receive them. It costs nothing when nobody subscribes and it is always available, with no configuration and no extra process.

on returns an unsubscribe function. Call it on shutdown, or in tests, so handlers do not accumulate.

import { cache } from './index.js'

const off = cache.on((event) => {
  switch (event.type) {
    case 'hit':
      // level is 'L1' or 'L2'. That distinction is the whole point of two layers.
      metrics.increment('cache.hit', { level: event.level })
      break
    case 'miss':
      // level is 'L1', 'L2' or 'negative'.
      metrics.increment('cache.miss', { level: event.level })
      break
    case 'loader:success':
      metrics.timing('cache.loader.duration', event.durationMs)
      break
    case 'inflight:reuse':
      metrics.increment('cache.inflight.reuse')
      break
    case 'set:received':
      // A peer's broadcast warmed this server's L1 for free.
      metrics.increment('cache.set.received')
      break
  }
})

process.on('SIGTERM', () => off())

The four numbers worth deriving from this stream first are your hit ratio, your L1 to L2 hit split, your loader duration distribution, and your in-flight reuse count. Together they tell you whether L1 is sized right, whether L2 is earning its network hop, and whether herds are collapsing.

The dashboard

A live view of counters, an L1 and L2 key explorer with per-key serialized versus decoded sizes, an event feed over server sent events, and the resolved configuration.

import { LazyLayersCache, RedisStore } from 'lazy-layers-cache'
import { redis } from './redis.js'

export const cache = new LazyLayersCache({
  ttlMs: 5 * 60 * 1000,
  l2: new RedisStore(redis, { prefix: 'app:cache:' }),

  observability: true,
})

That starts a standalone server bound to 127.0.0.1 on port 7077 and serves the dashboard at /__lazylayers, behind basic auth with the default credentials lazydev and lazydev. So the URL you open is:

http://127.0.0.1:7077/__lazylayers

Turning it on prints a one time warning to console.warn, deliberately outside the gated logger, because production is exactly where you do not want this left running.

Configuring it

export const cache = new LazyLayersCache({
  ttlMs: 5 * 60 * 1000,
  l2: new RedisStore(redis, { prefix: 'app:cache:' }),

  observability: {
    enabled: true,
    route: '/internal/cache',
    // Ring buffer for the live feed. Held in memory, never written to Redis or disk.
    maxEvents: 5_000,
    // Decoded values larger than this are truncated in the UI.
    maxValueBytes: 64 * 1024,
    auth: {
      username: 'ops',
      password: process.env.CACHE_DASH_PASSWORD,
    },
    server: { host: '127.0.0.1', port: 7077 },
    // Silence the one-time startup notice once you have read it.
    quiet: true,
  },
})

Credentials from the environment

Every dashboard setting also reads an environment variable, so you can run it in staging without a code change. Precedence is option, then environment variable, then default.

LAZY_OBS_ENABLED=1
LAZY_OBS_ROUTE=/internal/cache
LAZY_OBS_HOST=127.0.0.1
LAZY_OBS_PORT=7077
LAZY_OBS_USER=ops
LAZY_OBS_PASSWORD=$CACHE_DASH_PASSWORD
LAZY_OBS_MAX_EVENTS=5000
LAZY_OBS_QUIET=1

The rest of the surface, including LAZY_OBS_TOKEN, LAZY_OBS_NO_SERVER, and LAZY_OBS_MAX_VALUE_BYTES, is in the environment variables reference.

Mounting it in your own server

Set server: false and mount the handler on a route your own middleware already protects. This is the only shape worth considering outside development, because it inherits your real authentication rather than adding a second, weaker one.

import express from 'express'
import { cache } from './cache/index.js'
import { requireAdmin } from './auth.js'

const app = express()
const handler = cache.getObservabilityHandler()

app.use('/internal/cache', requireAdmin, (req, res, next) => {
  // The handler returns true when it answered the request.
  if (!handler?.(req, res)) next()
})

process.on('SIGTERM', async () => {
  await cache.closeObservability()
})

getObservabilityHandler returns undefined when observability is off, which is why the call is optional chained.

Prometheus

The dashboard is for a human looking at one process. Prometheus is for the fleet.

export const cache = new LazyLayersCache({
  ttlMs: 5 * 60 * 1000,
  l2: new RedisStore(redis, { prefix: 'app:cache:' }),

  observability: {
    enabled: true,
    route: '/internal/cache',
    auth: { username: 'ops', password: process.env.CACHE_DASH_PASSWORD },
    prometheus: {
      enabled: true,
      prefix: 'lazycache',
    },
  },
})

Metrics are exposed at {route}/metrics, so with the route above that is /internal/cache/metrics. The prefix defaults to lazycache.

Every series is labelled by level, kind, or result only. No exported metric carries a cache key, which is what keeps the cardinality of this endpoint fixed no matter how many keys you cache.

If your scraper cannot present basic auth, prometheus: { public: true } leaves the metrics endpoint unauthenticated while the dashboard itself stays behind credentials. Reach for that only when the endpoint is already inside a network your scraper and nothing else can reach.

OpenTelemetry and APM

Raw events are published to a node:diagnostics_channel named lazycache:cache:event, which is the seam to attach a tracer to.

import { subscribeTelemetry } from 'lazy-layers-cache'

const unsubscribe = subscribeTelemetry((event) => {
  // Span attributes, not metric labels. A span attribute may carry the key.
  span.addEvent(event.type, { 'cache.level': String(event.level ?? '') })
})

The publish path is guarded by a hasSubscribers check, so with nothing attached it costs one boolean test on the hot path.

The historical /observelazyily route remains an alias for the default dashboard.

Where to next