Every voice-pipeline LLM call (FlowAgent step, Vivoxx avatar, TUCAN Operator) routes through the Dudoxx LiteLLM gateway with thinkingBudgetTokens: 0, an OpenAI-compatible client wrapper, and tight latency budgets — so voice agents never hit OpenAI / Anthropic directly, never burn tokens on internal reasoning, and never leak the model choice into transport code.
Business Purpose
Voice is the most latency-sensitive surface in HMS — a 10 s "thinking" pause kills the conversation. It is also the most cost-sensitive (every silent re-prompt is a paid LLM round-trip) and the most policy-sensitive (the Mastra agent has access to PHI tools). Centralising every voice-side LLM call behind LiteLLM gives us:
One place to change models without redeploying ddx-livekit-flowagent-ts / ddx-vivoxx
Per-tenant quota + per-model budget enforcement
Forced enable_thinking: false (reasoning tokens MUST be 0 for voice — see Cardinal "default LLM" rule in root CLAUDE.md)
Strict provider whitelist (the Vivoxx LLM factory throws if anything but LITELLM / dudoxx is passed)
This page describes only the voice-side LLM contract. Text-side TUCAN dispatcher LLM routing is documented in tucan-dispatcher.md; the LiteLLM gateway infra (port + auth) lives at infra-litellm.md.
Audiences
Investor: Model substitution is a config change, not a deploy. Lets us race upstream model releases without code risk.
Clinical buyer: Voice answers stay under the latency floor; no foreign cloud sees PHI; clinic admin can cap monthly LLM spend in LiteLLM dashboard.
Developer/partner: Single createDudoxxMastraModel({ baseURL, apiKey, model }) factory; everything else (timeouts, retries, thinking-off injection) is handled by litellmFetch.
Internal (ops/support): One env var (DUDOXX_BASE_URL) controls every voice-side LLM call. One token (DUDOXX_API_KEY) gates them all. One model name (DUDOXX_MODEL_NAME) is the default.
The voice-side caller never knows whether the upstream model is local vLLM or cloud. It just picks an alias (dudoxx-gemma, openai/gpt-4o, etc.) and trusts the gateway to route.
Tech Stack & Choices
Client SDK: @ai-sdk/openai v5 with a custom createLitellmFetch middleware (ddx-livekit-flowagent-ts/src/mastra/model.ts:18-103) that intercepts request bodies and injects enable_thinking:false, temperature, thinkingBudgetTokens, maxOutputTokens.
Endpoint policy: forces provider.chat(model) → /v1/chat/completions (Mastra default would hit /v1/responses which LiteLLM does not support — model.ts:135-138).
Vivoxx side: same provider whitelist (ddx-vivoxx/src/providers/llm.provider.tsaccepted = ['dudoxx','DUDOXX','LITELLM','litellm','DUDOXX_LITELLM','DUDOXX_LLM'] — anything else throws); uses @livekit/agents-plugin-openai LLM with the same baseURL.
TUCAN side: same gateway, different factory (ddx-api/src/ai/tucan/registry/llm-config.factory.ts:316 documents DUDOXX_BASE_URL as the LiteLLM proxy URL). Upstream probe (dudoxx-upstream-probe.service.ts:108-113) discovers each alias' real model name; falls back to regex if probe is disabled.
Default model: dudoxx-gemma ($1 / M tokens — INTERNAL_COST tier per root CLAUDE.md). All other tiers are ZERO_COST.
Thinking budget: 0 for voice (MEMORY_LIVEKIT.md "thinkingBudgetTokens: 0 for voice"). Reasoning tokens MUST be 0 — verified at observability level.
Why not model: inherit? It IS the rule for Mastra Agent() calls (root project memory feedback_agent_model_inheritance.md) — voice agents declare the model once at workflow-compile time and pass it into every Agent factory call (workflow-compiler.ts:101, :126, :139).
Why not call providers directly? Three reasons: (1) latency monitoring, (2) per-tenant budget, (3) compliance log of every PHI-adjacent prompt. Direct OpenAI / Anthropic / vLLM is rejected at provider whitelist time.
Data Flow
FlowAgent voice step starts: workflow-compiler.ts produces a step that calls a Mastra Agent with model: createDudoxxMastraModel({ baseURL: config.llm.baseUrl, apiKey: config.llm.apiKey, model: config.llm.model, thinkingBudgetTokens: 0 }).
The Mastra agent receives TranscriptionFinal.text (from STT) plus prior memory and the step's tool set (≤7 voice-essential tools — see MEMORY_LIVEKIT.md "7 essential tools").
The @ai-sdk/openai client serialises the chat request; litellmFetch (mastra/model.ts:18-103) intercepts, parses the JSON body, injects temperature, thinkingBudgetTokens, maxOutputTokens, and enable_thinking: false, then forwards via globalThis.fetch to DUDOXX_BASE_URL.
LiteLLM resolves the alias, calls the upstream provider, returns the OpenAI-compatible response.
Mastra parses tool calls; voice-allowed tools execute (fill_field, skip_field, batch_fill, …); the assistant text is sanitised (strip <think>, "section is complete" hallucinations, markdown) before being handed to the TTS provider (Dudoxx CUDA Live TTS; Kokoro is retired).
Observability: LiteLLM logs wall_ms, input_tokens, output_tokens, reasoning_tokens (must be 0), cost_usd. Same metrics block format as rag_investigate (root CLAUDE.md).
Implicated Code
MANDATORY: ≥3 file:line citations.
ddx-livekit-flowagent-ts/src/config.ts:21-25 — single source of truth for voice-pipeline LLM config (baseUrl, apiKey, model); defaults to http://localhost:4000/v1 (LiteLLM dev) and openai/gpt-4o.
ddx-livekit-flowagent-ts/src/mastra/model.ts:122-139 — createDudoxxMastraModel factory; forces provider.chat(model) to hit /v1/chat/completions (LiteLLM does not implement /v1/responses).
ddx-livekit-flowagent-ts/src/mastra/model.ts:18-103 — createLitellmFetch middleware: parses request body, injects enable_thinking:false / temperature / thinking budget, passes through to globalThis.fetch.
ddx-vivoxx/src/providers/llm.provider.ts:1-20 — Vivoxx LLM provider preamble: "Direct OpenAI cloud is rejected — ALL LLM calls route through LiteLLM."
ddx-vivoxx/src/providers/llm.provider.ts:36-55 — provider whitelist + openaiPlugin.LLM construction with baseURL: config.llm.baseUrl.
ddx-api/src/ai/tucan/registry/llm-config.factory.ts:316 — DUDOXX_BASE_URL documented as LiteLLM proxy URL.
ddx-api/src/ai/tucan/registry/dudoxx-upstream-probe.service.ts:108-113 — probe disabled when DUDOXX_BASE_URL/DUDOXX_API_KEY missing → regex fallback.
ddx-livekit-flowagent-ts/src/mastra/workflow-compiler.ts:101, :126, :139 — step model parameter wiring.
Operational Notes
Gateway port: see infra-litellm.md for the production port + auth. The voice services use DUDOXX_BASE_URL (default http://localhost:4000/v1 in dev); the W4 brief mentioned port 15800 — verify against current docker-compose.yml before assuming that number is canonical.
Provider whitelist: anything other than the accepted set in ddx-vivoxx/src/providers/llm.provider.ts throws. Updating must update the whitelist AND the LiteLLM config.
enable_thinking: false rule: enforced at three layers — gateway default (root CLAUDE.md M-USER 2026-05-17), litellmFetch body injection (model.ts), and voice-FSM rule (MEMORY_LIVEKIT.md). If reasoning_tokens > 0 shows up in LiteLLM analytics for a voice session, it is a regression in the middleware or the gateway default.
Latency target: <1 s first-token for voice. Increasing thinkingBudgetTokens blows this budget — measured 10-21 s for 1024 tokens (MEMORY_LIVEKIT.md).
Model substitution: change DUDOXX_MODEL_NAME env or per-call override; do not edit the consumer code.
Pitfall — Mastra default endpoint: AI SDK v5 provider() calls /v1/responses (OpenAI Responses API), which LiteLLM does not support. ALWAYS use provider.chat(model) (model.ts:135-138). This is the most common cause of "LiteLLM 404" voice regressions.
Pitfall — Groq 503 with reply_to_operator: TUCAN Operator-side defect (MEMORY_LIVEKIT.md Known Open Bugs) — when a voice tool is not in request.tools, some upstream providers 503. Mitigation: always inject the voice tool set explicitly.