Redis — Cache and SSE Transport

Redis — Cache and SSE Transport

Redis is the platform's nervous system: it caches hot reads, holds short-TTL appointment-slot reservations, and — most importantly — serves as the durable transport (Redis Streams) and live fan-out (pub/sub) for every Server-Sent Event the platform broadcasts to ddx-web.

Business Purpose

Two clinic-facing problems converge on Redis. First, real-time UI: when a doctor finishes an encounter, the patient's portal needs to see the prescription appear within a second; when reception books a slot, every staff calendar must reflect the change immediately. Second, contention: two staff members must not simultaneously book the same appointment slot. Redis solves both — Streams give us a durable, ordered event log that ddx-web replays after reconnect (so a brief network blip does not silently lose updates), and short-TTL reservations give us optimistic slot locking without a heavy distributed-lock service. The result is a UI that feels live without sacrificing the audit trail or the ability to scale ddx-api horizontally.

Audiences

• Investor: Redis is the multi-instance scaling foundation. The SSE design with Redis Streams + pub/sub means we can run ddx-api on several nodes behind a load balancer, and any browser connected to any node will see every event — a precondition for shipping HMS to multi-clinic SaaS deployments.
• Clinical buyer (doctor/nurse/receptionist): The "live" feel of the UI — appointments updating across staff devices, prescriptions appearing in patient portals, voice-agent prompts appearing as the agent generates them — is mostly Redis underneath.
• Developer/partner: Two namespaces matter most. (1) CacheService (ddx-api/src/platform/cache/cache.service.ts:45) is the wrapped ioredis client used by feature modules for caching + slot reservation. (2) SseRedisTransport (sse-redis.transport.ts:42) is the only authorized publisher/subscriber seam for SSE — feature code calls RedisSsePublisher.publish(), never redis.publish() directly.
• Internal (ops/support): Single Redis 7 container on host :6379 (REDIS_PORT, 6xxx dev band; compose default 15379 is overridden by .env.docker), password-protected, AOF persistence enabled, 2 GB maxmemory with allkeys-lru eviction. Healthcheck pings every 10s with the configured password.

Architecture

On the ddx-hms-network, container ddx-001-redis-1 (redis:7-alpine, password-protected):

Three consumer paths reach it:

No direct redis calls exist outside these seams.

Two-channel design (Streams + pub/sub) is canonical — see redis-sse-publisher.ts:41-66 for the comment explaining the invariant logged ≤ delivered. Streams (XADD) are the durable, replayable delivery; pub/sub PUBLISH is for live PSUBSCRIBE listeners only. ddx-web reads via XREAD, so pubsubSubscribers=0 is the expected steady state and is not a delivery failure.

Architecture anchor: coding_context/ddx-hms-context.md for the broader HMS topology. The canonical SSE wire contract is packages/ddx-sse-contract/ (@ddx/sse-contract, 4.1.0) — this page describes the infrastructure, not the contract.

Tech Stack & Choices

• Engine: redis:7-alpine (docker-compose.yml:16). Pinned major version 7 because Streams and XADD/XREAD semantics are version-sensitive.
• Client: ioredis throughout NestJS. Chosen over node-redis for its mature pub/sub semantics, automatic reconnection, and duplicate() API (essential for SSE — subscriber connections cannot share with normal commands).
• Cache abstraction: NestJS Cache Manager (@nestjs/cache-manager) wraps ioredis underneath; CacheService uses both the manager (for typed get/set) and a raw redisClient (for setNx, keys, scan) — see cache.service.ts:48-52.
• Persistence: AOF + RDB (the config defaults documented in ddx-docker-redis/CLAUDE.md): save 900 1, save 300 10, save 60 10000, appendonly yes, appendfsync everysec. Trade-off: at most one second of data loss on a crash — acceptable for cache/queue workloads.
• Memory policy: maxmemory 2gb, maxmemory-policy allkeys-lru. Hot data stays in RAM; cold keys are evicted. Slot reservations carry explicit TTL so they are never evicted mid-reservation.
• Auth: password-protected (default password is in docker-compose.yml:30 for dev — rotate in production). Every healthcheck and client connection passes -a <password>.
• Why not Memcached / Hazelcast / NATS: Memcached lacks Streams and pub/sub; Hazelcast is a JVM-first ecosystem; NATS would have meant a second piece of infrastructure for events alongside Redis cache. Single store, two roles, one container.

Data Flow

SSE publish path (every server-side event the browser ever sees):

1. A feature service finishes its work and calls RedisSsePublisher.publish(channel, event) — the public facade is SsePublisherBase; this is the only authorized seam.
2. RedisSsePublisher.transportPublish() runs the invariant: XADD first, then PUBLISH — see redis-sse-publisher.ts:37-66. SseEventLogService.append() writes the envelope to a Redis Stream (durable, replayable). Then SseRedisTransport.publish() issues PUBLISH for any live listeners.
3. SseRedisTransport.publish() is at sse-redis.transport.ts:107-110 — a thin wrapper around ioredis.publish(channel, JSON.stringify(envelope)). Receivers count is logged; zero receivers is normal because ddx-web does not subscribe via pub/sub — it reads via XREAD.
4. High-volume token-streaming events (text:chunk, text:delta, plan:task:text:chunk, plan:task:text:delta) bypass the debug log line — see the HIGH_VOLUME_EVENTS set at redis-sse-publisher.ts:69-74.

SSE consume path (ddx-web GET):

1. Browser opens EventSource('/api/sse/stream?…') against Next.js, which proxies to SseController.stream() (ddx-api/src/platform/sse/sse.controller.ts:91-198).
2. The controller subscribes to the requested channels via SseRedisTransport.subscribe(channel, handler) (sse-redis.transport.ts:118-158). The transport uses a single dedicated subscriber ioredis client duplicated off the publisher in onModuleInit() (sse-redis.transport.ts:50-82).
3. On every PUBLISH, the subscriber callback invokes registered handlers; the controller writes the envelope to the HTTP response stream.
4. Reconnection: ddx-web sends Last-Event-Id; the controller replays missed events from the Redis Stream via runReplayThenSubscribe (sse.controller.ts:204-245) before joining live pub/sub — closing the "blip drops events" gap.

Cache + reservation path (slot booking):

1. Feature service calls CacheService.setReservation(...) (cache.service.ts:231) — writes a TTL-bound key.
2. Conflict checks call CacheService.getReservation(...) (cache.service.ts:247); confirms call confirmReservation() (:309).
3. Eviction is policy-driven (allkeys-lru) but reservation keys carry explicit TTLs so they expire deterministically.

Implicated Code

≥3 file:line citations required. Counts: 14 below.

• ddx-docker-redis/docker-compose.yml:15 — redis service definition.
• ddx-docker-redis/docker-compose.yml:23 — host port mapping ${REDIS_PORT:-15379}:6379; .env.docker sets REDIS_PORT=6379 (live 6xxx band, ddx-001 deployment).
• ddx-docker-redis/docker-compose.yml:30 — redis-cli ping healthcheck with password.
• ddx-api/src/platform/cache/cache.service.ts:45 — CacheService class (lines 45–483).
• ddx-api/src/platform/cache/cache.service.ts:55 — getRedis() lazy initializer (lines 55–84).
• ddx-api/src/platform/cache/cache.service.ts:231 — setReservation() write path for short-TTL slot reservations.
• ddx-api/src/platform/cache/cache.service.ts:247 — getReservation() read path for conflict detection.
• ddx-api/src/platform/cache/cache.service.ts:309 — confirmReservation() promote-to-permanent path.
• ddx-api/src/platform/sse/sse-redis.transport.ts:42 — SseRedisTransport class (lines 42–166), the only direct pub/sub seam.
• ddx-api/src/platform/sse/sse-redis.transport.ts:50 — onModuleInit() duplicates the publisher into a dedicated subscriber client (ioredis requirement).
• ddx-api/src/platform/sse/sse-redis.transport.ts:107 — publish() (lines 107–110) — wraps ioredis.publish.
• ddx-api/src/platform/sse/sse-redis.transport.ts:118 — subscribe() (lines 118–158) — idempotent per-channel registration with unsubscribe handle.
• ddx-api/src/platform/sse/redis-sse-publisher.ts:27 — RedisSsePublisher extends SsePublisherBase (lines 27–66).
• ddx-api/src/platform/sse/redis-sse-publisher.ts:37 — transportPublish() enforces the XADD before PUBLISH invariant.
• ddx-api/src/platform/sse/redis-sse-publisher.ts:69 — HIGH_VOLUME_EVENTS exclusion list (token-streaming events that bypass the debug log).
• ddx-api/src/platform/sse/sse.controller.ts:91 — stream() HTTP entry point (lines 91–198) for ddx-web SSE consumers.
• ddx-api/src/platform/sse/sse.controller.ts:204 — runReplayThenSubscribe() (lines 204–245) — replays missed events from Redis Streams before joining live pub/sub.

Business → Technical match. Business outcome: a doctor finishes a visit; the patient's portal shows the new prescription within a second, even if the doctor's browser briefly disconnected during the click. Technical mechanism: RedisSsePublisher.transportPublish() (redis-sse-publisher.ts:37) writes the envelope to a Redis Stream first (SseEventLogService.append()), so the event is durable even if no subscriber is currently listening; then SseRedisTransport.publish() (sse-redis.transport.ts:107) fans out via pub/sub for live listeners. When the patient's browser reconnects, sse.controller.ts:204 replays from the stream cursor — no event is silently lost.

Operational Notes

• pubsubSubscribers=0 in logs is NORMAL — it means no consumer is currently PSUBSCRIBE'd via pub/sub. ddx-web reads via XREAD (Streams), so the count is structurally zero. This was a recurring misread before the log message was renamed in redis-sse-publisher.ts:52-54; do not interpret it as event loss.
• Subscriber must be a duplicated connection. ioredis pub/sub requires a dedicated connection; mixing pub/sub with normal commands on one client is undefined behaviour. The transport does this at sse-redis.transport.ts:54 and tears it down in onModuleDestroy at :85-96.
• High-volume events bypass debug logging. Token-streaming chunks fire hundreds of times per LLM stream. The exclusion list is at redis-sse-publisher.ts:69-74; envelope-level events (start/complete/tool/plan/state) still log.
• Forbidden patterns (from ddx-api/CLAUDE.md):
  • Raw `agent:session-${id}` channel strings → always use SSEChannels.session(id) from @ddx/sse-contract.
  • SSEChannels.organization(organizationId) with a UUID → channels are keyed by clinic slug, not UUID (§9.3 of the SSE contract).
  • _meta: { ... } wrappers in publisher payloads → retired; v3 SseEnvelope top-level payload fields only.
• AOF persistence is on. Restart loses at most one second of data (appendfsync everysec). Memory cap 2 GB with allkeys-lru — cold keys evict first.
• Password rotation: change requirepass in config/redis.conf, rotate the matching ioredis config in ddx-api, restart both. The healthcheck password at docker-compose.yml:30 must match.
• Validation commands:
  • docker exec ddx-001-redis-1 redis-cli -a <pw> ping → PONG
  • docker exec ddx-001-redis-1 redis-cli -a <pw> info memory
  • docker exec ddx-001-redis-1 redis-cli -a <pw> XLEN <stream-key> — verify stream depth

Related Topics

• PostgreSQL — Multi-Schema Database Infrastructure — the durable system-of-record. Redis Streams are the event-log layer above it: Postgres writes commit first, the SSE event publishes second.
• LiveKit — WebRTC Media Server — when --profile production is enabled, LiveKit gets its own private Redis (port 15380) for session storage. That is a separate Redis from the HMS Redis documented here.
• Canonical SSE contract (not duplicated here): packages/ddx-sse-contract/ (@ddx/sse-contract, 4.1.0) — publisher/consumer rules, channel patterns, forbidden patterns.
• Full SSE event inventory: docs/sse-event-inventory.generated.md.