01-infrastructurewave: W1filled18 citations

Traefik — Reverse Proxy, TLS, and HMS Network Topology

Audiences: developer, internal

Traefik — Reverse Proxy, TLS, and HMS Network Topology

One Traefik v3 edge proxy with a Cloudflare-issued *.dudoxx.com wildcard cert routes every tenant subdomain over HTTPS; the multi-tenant compose stacks attach to a shared traefik-public Docker network so new tenants come online with zero per-tenant cert issuance.

This page covers both the Traefik docker configuration (W1-docker-traefik) and the inter-service networking topology / routing rules across the HMS (W1-networks-and-routing).

SCOPE NOTE: this page describes the production multi-tenant edge (ddx-docker-full-hms/ + ddx-docker-traefik/), whose containers use internal ports ddx-api:4100 / ddx-web:4000 / postgres-main:5432. That is a DIFFERENT deployment model from the local dev stack (.env.docker / ddx-docker-* single-service, host ports in the 6xxx band, ddx-001-* containers). Do not conflate the two: dev = 6xxx host ports + ddx-001-postgres-1; prod-edge = internal container hostnames behind Traefik. Ports below are the full-hms/edge model unless noted.

Business Purpose

Why Traefik (the proxy story)

  • One cert, infinite subdomains. A wildcard *.dudoxx.com certificate via Cloudflare DNS-01 ACME means provisioning a new tenant (acme.dudoxx.com, derm4hair.dudoxx.com, …) is just adding labels on the tenant compose stack — no new certbot run, no DNS-01 race per tenant.
  • Zero-config tenant onboarding. Every tenant's ddx-web declares routing labels; Traefik picks them up via the Docker socket provider and serves the host immediately.
  • Single observability surface — one access-log JSON stream, one rate-limit middleware, one HSTS policy.

Why the network topology matters (the routing story)

The five ddx-docker-* compose stacks (full-hms, traefik, jitsi, litellm, plus single-service helpers) share data through named external networks, not through host ports. This means:

  • ddx-api can reach Postgres / Redis / Qdrant / MinIO by hostname (postgres-main:5432, redis:6379) inside the default network without exposing them to the LAN.
  • Only edge services (Traefik :443, Jitsi JVB :15100/udp, optional voice ports) are bound to host interfaces.
  • Tenants are isolated by Docker Compose project (-p <trigram>); each gets its own subnet (/16) from manifest.network.subnet.

Audiences

  • Investor: a wildcard cert + DNS-01 + Docker-labels routing is the cheapest possible multi-tenant onboarding path — no ops cost per tenant cert, no per-tenant DNS rotation. This is what makes "add a clinic on Monday, go live Tuesday" credible.
  • Clinical buyer (doctor/nurse/receptionist): invisible — they see HTTPS in the address bar.
  • Developer/partner: copy the label block from ddx-docker-traefik/README.md onto any new service to publish it; everything else is automatic.
  • Internal (ops/support): acme.json MUST be chmod 600 or Traefik refuses to start. Dashboard is on 127.0.0.1:8080 only (basic-auth gated via dynamic.yml).

Architecture

Edge proxy

diagram
Internet ──:443── Traefik v3 (1 instance) ──┬── ddx-web (tenant A)  on traefik-public
                       │                    ├── ddx-web (tenant B)  on traefik-public
                       │                    ├── llm-router         (LiteLLM, optional)
                       │                    ├── meet-tucan         (Jitsi web)
                       │                    └── traefik.dudoxx.com  (dashboard)
              cert via Cloudflare DNS-01 (*.dudoxx.com)
  • One Traefik container, project name traefik, started with docker compose -p traefik -f ddx-docker-traefik/docker-compose.yml up -d.
  • Two entry points: web (:80, hard-redirects to :443) and websecure (:443, mandatory TLS).
  • Wildcard cert obtained via the Cloudflare DNS-01 challenge — works even when port 80 is blocked or behind a home router.
  • Optional cloudflare-ddns sidecar (--profile dns-auto) keeps the wildcard A-record pointed at the dynamic public IP — exactly one shared instance, never per-tenant (DDNS clients race on Cloudflare API).

Multi-stack network topology (the routing story)

diagram
┌─────────────────────── traefik-public (external, shared) ─────────────────┐
│   Traefik :443   ─►   ddx-web (acme)   ─►   ddx-web (d4h)   ─►   ...      │
└─────────┬────────────────────┬───────────────────┬────────────────────────┘
          │                    │                   │
   ┌──────┴──────┐      ┌──────┴──────┐     ┌──────┴──────┐
   │ acme stack  │      │ d4h stack   │     │ jitsi stack │
   │ (project    │      │ (project    │     │ (project    │
   │  -p acme)   │      │  -p d4h)    │     │  -p jitsi)  │
   │             │      │             │     │             │
   │ default:    │      │ default:    │     │ ddx-jitsi:  │
   │  10.20/16   │      │  10.30/16   │     │ bridge      │
   │             │      │             │     │             │
   │ ddx-api ↔   │      │ ddx-api ↔   │     │ web↔prosody │
   │ postgres ↔  │      │ postgres ↔  │     │ ↔ jicofo ↔  │
   │ redis ↔ ... │      │ redis ↔ ... │     │ jvb ↔ jibri │
   └─────────────┘      └─────────────┘     └─────────────┘

Two classes of network:

  1. Per-tenant default network (one per docker compose -p <trigram>) — internal app traffic: postgres-main, redis, qdrant, minio, hapi-fhir, ddx-api, ddx-web all resolve each other by container hostname on this subnet. Tenant subnet comes from manifest.network.subnet (default 10.100.0.0/16).
  2. Shared traefik-public networkexternal: true, created once with docker network create traefik-public. Only ddx-web (and optionally hapi-fhir if exposed) attaches to BOTH networks; everything else stays on the per-tenant default.

This is what gives multi-tenant isolation without per-tenant Traefik configuration:

  • Tenant A's ddx-api cannot reach Tenant B's postgres — different subnets, no shared network.
  • All tenants' ddx-web are reachable through one Traefik instance — the traefik-public network is the only bridge.

Internal DNS / hostname resolution

Within a -p <trigram> project, Docker's embedded DNS resolves service names directly:

Caller (ddx-api) → calleeResolved hostname
Main Postgres (6 DBs)postgres-main:5432
HAPI FHIRhapi-fhir:8080
Redisredis:6379
Qdrantqdrant:6333
MinIOminio:9000
Jitsi web (cross-bridge)jitsi-web:80
LiveKitlivekit:36880 (within livekit-rendered-config)
ddx-web (server-side fetch)http://ddx-web:4000

Host-side port mappings exist for development only — production traffic stays internal via the container hostnames above. In the full-hms compose the dev host ports default to 15432/etc. (PORT_POSTGRES_MAIN); the standalone local dev stack (ddx-docker-* + .env.docker) instead binds the 6xxx band (postgres 6432, redis 6379, qdrant 6333, minio 6000) on ddx-001-* containers — see context/ENVIRONMENT.md.

Tech Stack & Choices

  • Traefik v3.2 — chosen over Nginx + certbot because (a) Docker label-driven routing fits the multi-tenant model perfectly; (b) DNS-01 ACME is a first-class plugin; (c) built-in dashboard + rate-limit + headers middlewares replace 200 lines of Nginx + ufw + fail2ban; (d) v3 brought stable HTTP/3 + multi-arch.
  • Cloudflare DNS-01 over HTTP-01 — works behind NAT / when port 80 is filtered; necessary for home-server deployments. Provider switch is one line in traefik.yml.
  • Wildcard cert (dudoxx.com + *.dudoxx.com SAN) — eliminates per-tenant ACME order; one renewal every 60 days covers every current and future subdomain.
  • acme.json storage at chmod 600 — Traefik enforces this on startup (writes private key material). Forgetting this is the #1 first-time failure.
  • Dashboard bound to 127.0.0.1:8080 + basic-auth via dynamic.yml (htpasswd -nbB) — never exposed publicly; access through SSH tunnel or routed through traefik.dudoxx.com with the same basic-auth.
  • TLS 1.2 minimum, sniStrict: true (dynamic.yml) — no SSL 3.0 / TLS 1.0/1.1, no SNI-less request handling (prevents some scanner classes).
  • HSTS preload + frameDeny middleware available globally as secure-headers@file; per-route opt-in.

Data Flow

Request path (browser → tenant ddx-web)

  1. Browser resolves acme.dudoxx.com via Cloudflare DNS → public IP (DDNS keeps this fresh).
  2. TCP :443 → host firewall → Traefik container (mapped 443:443).
  3. Traefik terminates TLS (wildcard cert from acme.json), reads the Host header.
  4. Docker provider matches Host(\acme.dudoxx.com`)to the router declared onddx-web` labels.
  5. Traefik forwards over traefik-public to ddx-web:4000 (Next.js — full-hms internal container port; the dev stack instead serves ddx-web on host :6200).
  6. ddx-web SSR-fetches ddx-api via http://ddx-api:4100/api/v1 on the per-tenant default network (NOT traefik-public). (Dev-stack equivalent: http://localhost:6100/api/v1.)

Cert renewal

  • Every 60 days, Traefik triggers a new ACME order with Cloudflare DNS-01.
  • The CF_DNS_API_TOKEN (scoped Zone.DNS:Edit on dudoxx.com) places a TXT record; delayBeforeCheck: 10s lets it propagate.
  • New cert lands in acme.json; live config swaps in without a restart.

New tenant onboarding (the "go-live Tuesday" claim)

  1. Create tenant-data/<trigram>/manifest.json (ports, subnet, externals).
  2. provision-tenant.sh brings up the stack with -p <trigram>, attaching ddx-web to traefik-public and to default.
  3. Traefik's Docker provider sees the new labels and starts serving <trigram>.dudoxx.com — covered by the existing wildcard cert. No certbot, no DNS change (if DDNS sidecar is running).

Implicated Code

MANDATORY: ≥3 file:line citations.

  • ddx-docker-traefik/docker-compose.yml:15traefik service, image: traefik:v3.2.
  • ddx-docker-traefik/docker-compose.yml:19 — host port bindings "80:80" and "443:443"; dashboard at "127.0.0.1:8080:8080" (local-only).
  • ddx-docker-traefik/docker-compose.yml:26CF_DNS_API_TOKEN env (Zone.DNS:Edit scope).
  • ddx-docker-traefik/docker-compose.yml:31 — mounts /var/run/docker.sock:/var/run/docker.sock:ro (provider) and acme.json:/etc/traefik/acme/acme.json.
  • ddx-docker-traefik/docker-compose.yml:36networks: - traefik-public (the shared external network).
  • ddx-docker-traefik/docker-compose.yml:64 — optional cloudflare-ddns sidecar (one per platform, profile dns-auto).
  • ddx-docker-traefik/docker-compose.yml:77traefik-public network declared external: true.
  • ddx-docker-traefik/traefik.yml:29entryPoints.web HTTP→HTTPS permanent redirect.
  • ddx-docker-traefik/traefik.yml:37entryPoints.websecure default certResolver cloudflare with main: dudoxx.com + sans: *.dudoxx.com.
  • ddx-docker-traefik/traefik.yml:50providers.docker with exposedByDefault: false (services must opt in via label) + network: traefik-public.
  • ddx-docker-traefik/traefik.yml:60certificatesResolvers.cloudflare.acme.dnsChallenge with resolvers 1.1.1.1:53, 1.0.0.1:53 and delayBeforeCheck: 10.
  • ddx-docker-traefik/dynamic.yml:16dashboard-auth basicAuth middleware (htpasswd hash from env).
  • ddx-docker-traefik/dynamic.yml:24default-ratelimit (50 avg / 100 burst / 1s).
  • ddx-docker-traefik/dynamic.yml:33secure-headers (HSTS preload, frameDeny, no-sniff).
  • ddx-docker-traefik/dynamic.yml:45 — TLS options minVersion: VersionTLS12, sniStrict: true.
  • ddx-docker-full-hms/docker-compose.yml:1228 — per-tenant default bridge network with subnet from ${TENANT_SUBNET:-10.100.0.0/16}.
  • ddx-docker-full-hms/docker-compose.yml:1236traefik-public declared external: true (links into the edge).
  • ddx-docker-full-hms/docker-compose.yml:81postgres-main host port ${PORT_POSTGRES_MAIN:-15432}:5432 (dev only; production traffic is internal postgres-main:5432).

Operational Notes

  • acme.json permissions: must be chmod 600 before first start. Traefik refuses to start otherwise. Symptom: the ACME resolver "cloudflare" is skipped from the resolvers list because: unable to get ACME store in logs.
  • traefik-public network bootstrap: docker network create traefik-public is a one-time, manual step. If you docker network rm traefik-public, every running tenant stack loses its edge connection silently — re-create + restart.
  • Cloudflare token scope: ONLY Zone.DNS:Edit on dudoxx.com. A broader token is a HIPAA exposure (token compromise = arbitrary Cloudflare action).
  • Wildcard cert covers one level only: *.dudoxx.com does NOT cover api.acme.dudoxx.com. For nested subdomains, add *.*.dudoxx.com to the sans list AND issue a second wildcard via DNS-01.
  • Dashboard discovery pitfall: dashboard router uses Host(\traefik.dudoxx.com`) — but the basic-auth middleware reads the htpasswd hash from env (TRAEFIK_DASHBOARD_PASSWORD_HASH). Escape $as$$` if hashed value goes into compose directly.
  • DDNS sidecar is opt-in (--profile dns-auto). Running multiple DDNS clients (one per tenant) races on Cloudflare API rate-limits — DO NOT replicate.
  • Inter-stack routing: there is no NAT between stacks. To call across stacks (e.g. ddx-api in tenant A → litellm in a separate stack), the destination must be on the same shared network OR be reached via Traefik over the public hostname. The current pattern is: LiteLLM is reached via https://llm-router.dudoxx.com/v1 (back through Traefik), Jitsi by per-tenant jitsi-web:80 (same compose project).
  • Validation:
    bash
    curl -k --resolve acme.dudoxx.com:443:127.0.0.1 https://acme.dudoxx.com/healthz
    docker logs traefik --tail 50 | grep -E "Register|Server is up"
    docker network inspect traefik-public --format '{{json .Containers}}' | jq
    
  • infra-jitsi — published as meet-tucan.dudoxx.com via the same Traefik
  • infra-litellm — published as llm-router.dudoxx.com via the same Traefik
  • infra-livekit — RTC ports bypass Traefik (UDP / different entrypoint)
  • infra-full-hms-compose — umbrella that declares the per-tenant default network and attaches ddx-web to traefik-public
  • infra-postgres — host-port 15432 is dev-only; production traffic stays internal postgres-main:5432