12-operations-runbookswave: W6filled4 citations

DDX Install — Full-Stack Installer & Tenant Provisioner CLI

Audiences: internal, developer, partner

DDX Install — Full-Stack Installer & Tenant Provisioner CLI

ddx-install is a single Node CLI that brings the whole DDX HMS stack (8 infra services + the app tier + HAPI FHIR) up on a new server from a git clone. It is an orchestrator + wiring engine — it wraps the proven ddx-docker-full-hms scripts rather than reimplementing them — and its net-new value is a typed env-wiring engine that resolves the cross-module shared-key graph that otherwise fails silently.

Business Purpose

Standing up DDX HMS on a fresh host is hard not because any single step is hard, but because ~720 environment variables across 19 services collapse to ~36 shared logical keys, 17 of which are critical: a mismatch produces no error — just 401s, "Connecting…" hangs, or services that refuse to boot. ddx-install makes deployment reproducible and auditable: an operator (or an unattended JSON file) selects a trigram + port block, and the CLI mints secrets, wires every module's .env, validates the manifest, brings the compose stack up, migrates 5 schemas, seeds platform data, and prints the portal URL — gating on a parity check that catches the silent-mismatch class before any service starts.

This is the on-prem / partner-deployment workhorse. It is distinct from the in-API tenant-provisioner (which onboards a clinic inside an already-running stack); ddx-install brings the entire stack up.

Audiences

  • Internal (ops/deployment): One command (ddx-install unattended <file>) takes a clean host to a live, seeded, FHIR-backed stack. Resumable — re-running skips completed steps; --force re-runs all.
  • Developer: The wiring engine is the interesting part. wiring-graph.json is the data spine; wiring-engine.ts resolves it; execute.ts runs the 11-step pipeline and asserts 6 critical contracts before any mutation.
  • Partner: Deploy DDX HMS into customer-owned infrastructure with a typed, validated wiring layer instead of hand-edited env files.

Architecture

Rendering diagram…

The core principle (plan shard _index.md §3): wrap, do not reimplement. execute.ts shells out (via execa) to the existing validate-manifest.sh, generate-secrets.sh, manifest-to-env.sh, and docker compose — it never mints its own secrets or writes its own compose file.

Tech Stack & Choices

ConcernChoiceRationale
CLI frameworkcommander + @inquirer/prompts10-command tree + interactive main menu from one COMMANDS catalogue so the two never drift (cli.ts)
Shelling outexeca + ora spinnerswraps the canonical bash scripts; heavy logs delegated to tenant-data/<trigram>/.install-logs/ on failure
Data spinewiring-graph.jsonmachine-readable projection of the env journal: logicalKey → {producer, consumers:[{module, varName}], critical?, forcedValue?} — 36 logical keys, 17 critical edges, 14 consumer modules
Validationplain zod-free standalone CLIno NestJS; out() stdout pattern (no console.log)
Backbone (v1)Docker-first (ddx-docker-full-hms)PM2 bare-process is the documented v2 path (plan shard 40), not built this iteration
pnpm pin9.15.4 in image buildspnpm 10 breaks esbuild/sharp/prisma/@swc (Contract #3)

The 10 commands (cli.ts)

CommandArgsPurpose
doctorhost preflight: docker, disk, RAM, tool versions, ports, network
newattended wizard (clone → wire → provision → seed → verify)
up<trigram>(re)provision an existing tenant manifest
wire<trigram>regenerate per-module .env from graph+manifest (no provision)
seed<trigram>drive ddx-seeder-ts platform/org phases
status<trigram>compose ps + health curls + AC-7 key-parity
verify<trigram>full post-install verification suite
down<trigram>tear down (--volumes is destructive → double-confirm)
unattended<file.json>zero-prompt provision from tenant-unattended.json
graphprint/validate wiring-graph.json (debug the env edges)

Data Flow

Attended vs unattended convergence

Both modes resolve to one ResolvedTenantConfig, then converge on runExecute():

env-journal/*.md → wiring-graph.json (curated, typed)
tenant-unattended.json (or wizard) → ResolvedTenantConfig
ResolvedTenantConfig → toManifest()/toSecretsSeed() → manifest.json + secrets.json
                     → wiring-engine.wire() → per-module .env / .env.local

Secrets are never inlined: external provider keys use ENV:VARNAME indirection; the CLI mints all internal hex32 secrets itself.

The 11-step EXECUTE pipeline (execute.ts:runExecute)

Computed up front: the wire + recap + 6 contract checks run BEFORE any mutation. If any contract fails, the run aborts before touching docker.

  1. validate-manifest.sh (pre)
  2. generate-secrets.sh (idempotent — keeps real values)
  3. wiring-engine emit (per-module .env/.env.local, chmod 600)
  4. validate-manifest.sh (post) 4b. manifest-to-env.sh.env.<trigram> (the compose --env-file)
  5. docker compose -p ddx-hms-<trigram> up -d --build
  6. wait postgres-main healthy (pg_isready, 60s)
  7. db-init (migrations owned by the compose db-init service)
  8. wait ddx-api healthy (/api/v1/health, 60s)
  9. ddx-seeder profile container (org + FHIR partition + MinIO buckets + phases)
  10. optional profiles (voice / jitsi / vivoxx / calendar per capabilities)
  11. print portal URL + health summary

The 6 critical contracts (assertContracts)

  • C1 NEXT_PUBLIC-before-build — structural: wire (step 3) precedes compose build (step 5); the wiring-engine NEXT_PUBLIC guard rejects any secret-kind key emitted to a NEXT_PUBLIC_* var.
  • C2 flowagent amd64ddx-flowagent pinned linux/amd64 (onnxruntime native).
  • C3 pnpm 9.15.4 — pinned in all module Dockerfiles.
  • C4 AC-7 voice-key parityGATEWAY_API_KEY_VOICE (ddx-api) == DDX_API_KEY (ddx-flowagent), plus the other 16 critical edges.
  • C5 RBAC=enforceRBAC_ENFORCEMENT=enforce (graph forcedValue, and the ddx-docker-full-hms compose default ${RBAC_ENFORCEMENT:-enforce}). Only the bare application code-level fallback is warn, so the installer pins enforce explicitly.
  • C6 on-prem saas=falsemanifest.internal.saas=false → SAAS_ENABLED=false.

Implicated Code

  • ddx-install/src/cli.ts:49COMMANDS catalogue; drives commander registration + interactive menu
  • ddx-install/src/core/execute.ts:524runExecute(); the 11-step pipeline; assertContracts() at :254
  • ddx-install/src/core/execute.ts:495apiHealthy(); unwraps {data,meta} envelope (body.data?.status ?? body.status)
  • ddx-install/src/core/execute.ts:389 — authoritative 15-key internalKeys secrets contract (mirrors manifest-to-env.sh reads)
  • ddx-install/src/core/wiring-engine.tsloadWiringGraph() + wire(); parity + NEXT_PUBLIC guard
  • ddx-install/src/core/map-unattended.tstoManifest() / toSecretsSeed() (must satisfy canonical manifest.schema.json, additionalProperties:false)
  • ddx-install/data/wiring-graph.json — 36 logical keys, 17 critical edges, 14 consumer modules
  • ddx-docker-full-hms/scripts/provision-tenant.sh — the canonical 9-step provisioner the CLI wraps

Operational Notes

  • It wraps, it doesn't build. ddx-install never writes its own compose file or mints secrets — bugs in the underlying scripts surface as installer failures. Schema lives at the umbrella root tenant-data/_template/manifest.schema.json.
  • Docker-mode migrations are compose-owned. Step 7 is a no-op in Docker-first mode — the db-init compose service runs prisma migrate deploy ×5 in its own container. The host has no prisma toolchain; a host-side migrate would be redundant and broken (that path belongs to the v2 PM2 mode only).
  • 15-key secrets contract. manifest-to-env.sh reads exactly 15 keys from secrets.json (minioRootPassword, pgPassword, jwtSecret, …). Any missing key emits the literal string "null" into .env.<trigram> → the service FATALs at boot (e.g. minio "MINIO_ROOT_PASSWORD length should be at least 8").
  • AC-7 parity is the safety net. The recap shows PASS/FAIL for all 17 critical edges. Never trust a run that shows a FAIL — re-wire first.
  • NEXT_PUBLIC guard. Never emit a secret to a NEXT_PUBLIC_* var (known offender: NEXT_PUBLIC_KOKORO_TTS_TOKEN bakes a bearer token into the client bundle — correct pattern is server-only KOKORO_TTS_TOKEN + a TTS proxy route).
  • DDX_API_URL is a build-time edge. ddx-web's next build hard-throws "API_URL or DDX_API_URL must be set in production" — added to the graph as a Next.js logical key (docker → http://ddx-api:4100, host → http://localhost:<api port>).
    DDX Install — Full-Stack Installer & Tenant Provisioner CLI — Dudoxx Docs | Dudoxx