/Knowledge Base
GitHub
06-ai-tucanwave: W3filled29 citations

TUCAN Consumption Tracking and Pricing

Audiences: developer, internal, investor

TUCAN Consumption Tracking and Pricing — Per-Step Cost + Session Monitoring

Every assistant turn writes a typed TucanUsageRecord row with prompt tokens, completion tokens, reasoning tokens, BYOK flag, and a computed cost_usd. A pricing catalog refreshed at boot turns model slugs into per-token rates. The consumption REST surface rolls usage up per-session and per-user. Invariant: reasoning_tokens MUST be 0 when the model is dudoxx-gemma.

Business Purpose

Three audiences need accurate per-turn cost accounting, and none of them can wait for end-of-month vendor invoices:

  1. Finance / pricing team needs unit economics per session, per user, per tenant. Knowing "this clinic costs us $X / month in AI compute" is the input to the next price negotiation.
  2. Operators (Dudoxx + clinic admins) need a "did this session cost more than expected?" view. A regression in the prompt pipeline (e.g. accidental enable_thinking:true) will show up as a reasoning_tokens > 0 row.
  3. Product needs cohort analysis — does the new clinical agent cost less per outcome than the old one?

The TUCAN consumption layer ships the schema, the writer (per-step emit through the orchestrator's L5 persistency), the pricing catalog, and the rollup REST surface. The SSE observability service adds publish-side metrics (DLQ counts, publish failures) so the real-time stream itself is monitorable.

Audiences

  • Investor: Unit economics by design, not by report. Every assistant turn has its own row + cost. BYOK is flagged. Tenant isolation is enforced by organizationId on every record.
  • Internal (ops / finance): One SQL on tucan_usage_records GROUP BY model_slug answers "what's our internal vs. external model spend this week?"
  • Clinical buyer: Indirectly — the price they pay is informed by the actual COGS, not a guess.
  • Developer / partner: The append-only sibling TucanUsageRecord is the source of truth; do not aggregate from chat_messages. Cost-pricing rates live in dudoxx-pricing.service.ts, refreshed from the LiteLLM model catalog at boot.

Architecture

diagram
┌──────────────────────────────────────────────────────────────┐
│  Per-turn write (during L5 persistency)                      │
│   · Orchestrator persists ChatMessage{role:'assistant',      │
│       promptTokens, completionTokens, reasoningTokens,       │
│       costUsd, modelSlug, ...}                                │
│   · TucanUsageRecord row inserted 1:1 by chat_message_id     │
│       (append-only sibling; ON DELETE CASCADE)                │
│   · isByok flag set from MetricsMetadata.isByok               │
└──────────────────────────────────────────────────────────────┘
                              │
                              ▼  (REST + dashboards)
┌──────────────────────────────────────────────────────────────┐
│  ConsumptionController  @Controller('consumption')           │
│   GET  /me                       — own user rollup            │
│   GET  /sessions/:sessionId/usage — per-session breakdown     │
│                                                                │
│   Service: ConsumptionService                                  │
│     · groupBy(session, model, isByok) → SessionRollup          │
│     · per-row SessionUsage with token + cost                   │
│     · null cost preserved (unpriced ≠ free)                   │
└──────────────────────────────────────────────────────────────┘
                              │
                              ▼  (pricing layer)
┌──────────────────────────────────────────────────────────────┐
│  DudoxxPricingService  (OnModuleInit + OnModuleDestroy)       │
│   · refreshPricing() fetches model rates from LiteLLM         │
│   · computeCostUsd(modelSlug, promptTokens, completionTokens) │
│   · cost_usd = prompt × input_cost + completion × output_cost  │
│   · Unknown model → NULL cost (NOT zero)                      │
└──────────────────────────────────────────────────────────────┘
                              │
                              ▼  (SSE side telemetry)
┌──────────────────────────────────────────────────────────────┐
│  SseObservabilityService                                       │
│   · getPublishFailures() — count of failed SSE publishes      │
│   · getDLQCount(channel) / drainDLQ(channel)                   │
│   · getMetrics() — Prometheus exposition                       │
└──────────────────────────────────────────────────────────────┘

TucanUsageRecord schema (Prisma)

prisma
model TucanUsageRecord {
  chatMessageId    String   @id @map("chat_message_id")
  userId           String   @map("user_id")
  sessionId        String   @map("session_id")
  modelSlug        String?  @map("model_slug")
  promptTokens     Int?     @map("prompt_tokens")
  completionTokens Int?     @map("completion_tokens")
  reasoningTokens  Int?     @map("reasoning_tokens")
  costUsd          Decimal? @map("cost_usd") @db.Decimal(12, 6)
  isByok           Boolean? @map("is_byok")  -- forward-only; NULL for legacy
  createdAt        DateTime @default(now())  @map("created_at")
  updatedAt        DateTime @updatedAt        @map("updated_at")
}

Decimal(12,6) gives 6 decimal places of USD precision — enough for fractions-of-a-cent per-turn costs.

Null cost vs zero cost

The pricing service returns null for unknown models, NOT zero. The consumption service preserves null in rollups (it does not coerce to 0). This matters because:

  • "Unpriced" → operations should investigate (new model not in catalog).
  • "Free" → BYOK or internal-cost tier.

These are different states; collapsing them hides regressions.

SSE observability — different from cost telemetry

SseObservabilityService is the publisher-side health surface, not the cost surface. It tracks:

  • Publish failures (count of failed sends to Redis).
  • DLQ per channel (envelopes that failed terminally).
  • Prometheus exposition for scrape.

Cost lives on the turn-completion path; SSE observability lives on the send path. Both are part of "TUCAN monitoring."

Tech Stack & Choices

ChoiceRationale
Append-only TucanUsageRecord tableHot aggregation queries don't touch the wide ChatMessage row; cheap GROUP BY.
1:1 with ChatMessage by chat_message_id (PK + FK)Joinable for audit; CASCADE delete keeps integrity.
is_byok Boolean? forward-only columnLegacy rows are NULL (no backfill). Writers populate from MetricsMetadata.isByok; readers must tolerate NULL.
Per-row token columns (prompt/completion/reasoning)Native Postgres aggregation; no JSON parsing on read.
Decimal(12,6) for cost_usd6 decimal places = 0.000001 USD precision; integer math avoids float drift.
Pricing catalog refreshed at boot via LiteLLMSingle source for input/output rates; survives provider changes.
null cost preservedUnpriced ≠ free; rollups carry the null forward.
reasoning_tokens column on both ChatMessage and TucanUsageRecordDenormalized for cheap "did this turn think?" queries.
BYOK exclusion from vendor COGSWHERE is_byok IS NULL OR is_byok = false is the canonical filter.
SSE observability separate servicePublish health metrics are operationally distinct from cost.

Alternatives rejected:

  • Aggregate from ChatMessage directly — works at small scale, blows up at long sessions; bulk INSERT contention.
  • External pricing service (Stripe metering) — adds a hop; metering is owned data, not a SaaS dependency.
  • Float for cost_usd — float math drifts; Decimal is correct.

Data Flow

Business outcome: A clinic admin opens "Usage this month" in the portal and sees $47.23 across 412 sessions broken down by model and BYOK status.

Technical mechanism:

  1. Portal calls GET /api/v1/consumption/me?from=...&to=... (ddx-api/src/ai/tucan/consumption/consumption.controller.ts:46).
  2. ConsumptionService runs Prisma groupBy(modelSlug, isByok) on TucanUsageRecord for the user's organization (ddx-api/src/ai/tucan/consumption/consumption.service.ts:69).
  3. For each row, the service sums token columns and cost_usd preserving null where present (ddx-api/src/ai/tucan/consumption/consumption.service.ts:142).
  4. Response returns the rollup typed by UserRollupDto / SessionRollupDto.

Business outcome (publisher-side): A doctor reports "no stream events arrived for that one session yesterday." Ops queries getPublishFailures() + getDLQCount(channel) to find a Redis blip at 16:42, plus 3 DLQ envelopes which it then drains.

Technical mechanism:

  1. Prometheus scrape → SseObservabilityService.getMetrics() (ddx-api/src/ai/tucan/sse/observability/sse-observability.service.ts:293).
  2. Failure count is the getPublishFailures() counter (sse-observability.service.ts:177).
  3. DLQ drain returns the 3 failed envelopes (sse-observability.service.ts:187).

Implicated Code

  • ddx-api/prisma-ai/schema.prisma:101 — model TucanUsageRecord.
  • ddx-api/prisma-ai/schema.prisma:114 — is_byok forward-only column (NULL for legacy).
  • ddx-api/prisma-ai/schema.prisma:84 — ChatMessage.reasoningTokens
    • costUsd + modelSlug denormalized columns.
  • ddx-api/src/ai/tucan/consumption/consumption.controller.ts:34 — @Controller('consumption').
  • ddx-api/src/ai/tucan/consumption/consumption.controller.ts:35 — class ConsumptionController.
  • ddx-api/src/ai/tucan/consumption/consumption.controller.ts:46 — @Get('me') user rollup.
  • ddx-api/src/ai/tucan/consumption/consumption.controller.ts:72 — @Get('sessions/:sessionId/usage') per-session breakdown.
  • ddx-api/src/ai/tucan/consumption/consumption.service.ts:13 — null-cost semantics comment (unpriced ≠ free).
  • ddx-api/src/ai/tucan/consumption/consumption.service.ts:43 — reasoningTokens?: number DTO field.
  • ddx-api/src/ai/tucan/consumption/consumption.service.ts:46 — isByok?: boolean DTO field.
  • ddx-api/src/ai/tucan/consumption/consumption.service.ts:69 — class ConsumptionService.
  • ddx-api/src/ai/tucan/consumption/consumption.service.ts:103 — Prisma select { reasoningTokens, costUsd, … } for grouping.
  • ddx-api/src/ai/tucan/consumption/consumption.service.ts:142 — null-cost preserving rollup math.
  • ddx-api/src/ai/tucan/consumption/dto/usage-record.dto.ts — UsageRecordDto shape.
  • ddx-api/src/ai/tucan/consumption/dto/session-usage.dto.ts — per-session breakdown DTO.
  • ddx-api/src/ai/tucan/consumption/dto/session-rollup.dto.ts — per-session rollup DTO.
  • ddx-api/src/ai/tucan/consumption/dto/user-rollup.dto.ts — per-user rollup DTO.
  • ddx-api/src/ai/tucan/pricing/dudoxx-pricing.service.ts:5 — cost formula comment cost_usd = prompt × input + completion × output.
  • ddx-api/src/ai/tucan/pricing/dudoxx-pricing.service.ts:40 — class DudoxxPricingService implements OnModuleInit, OnModuleDestroy.
  • ddx-api/src/ai/tucan/pricing/dudoxx-pricing.service.ts:47 — onModuleInit() triggers initial catalog load.
  • ddx-api/src/ai/tucan/pricing/dudoxx-pricing.service.ts:77 — computeCostUsd(modelSlug, promptTokens, completionTokens).
  • ddx-api/src/ai/tucan/pricing/dudoxx-pricing.service.ts:91 — refreshPricing() (LiteLLM catalog fetch).
  • ddx-api/src/ai/tucan/sse/observability/sse-observability.service.ts:56 — class SseObservabilityService.
  • ddx-api/src/ai/tucan/sse/observability/sse-observability.service.ts:177 — getPublishFailures().
  • ddx-api/src/ai/tucan/sse/observability/sse-observability.service.ts:187 — drainDLQ(channel).
  • ddx-api/src/ai/tucan/sse/observability/sse-observability.service.ts:197 — getDLQCount(channel).
  • ddx-api/src/ai/tucan/sse/observability/sse-observability.service.ts:293 — getMetrics() Prometheus exposition.

Operational Notes

  • reasoning_tokens MUST be 0 when model is dudoxx-gemma. Project CLAUDE.md invariant: every chat/completions POST injects enable_thinking:false. A TucanUsageRecord row with reasoning_tokens > 0 means the thinking-off invariant has regressed; file a DEF. Same pattern is used by DDX RAG v3 cost-monitoring (reasoning_tokens asserted 0 in the metrics block).
  • Unpriced ≠ free. cost_usd = NULL means "unknown model" — refresh the pricing catalog or add the model to the rate table. Coercing NULL → 0 in dashboards hides regressions.
  • BYOK exclusion is WHERE is_byok IS NULL OR is_byok = false. Legacy rows (pre-2026-04 migration) are NULL; they were billable vendor cost at the time. Don't filter them out by default.
  • Decimal(12,6) is the contract. Six decimal places of USD is enough for fractions-of-a-cent per turn. Don't convert to float in business logic — round only at presentation.
  • Pricing catalog refresh. OnModuleInit loads it once; refreshPricing can be called by an admin endpoint or cron if the LiteLLM catalog changes mid-day. Mid-day rate changes are rare; bootstrap-only is acceptable.
  • tucan_usage_records is the source of truth for cost rollups. Aggregating from chat_messages is allowed for ad-hoc audit but slower (wider rows, no dedicated indexes).
  • Cascade delete. Removing a ChatMessage removes the matching TucanUsageRecord via ON DELETE CASCADE. Don't archive a single message; archive the whole session.
  • DDX RAG v3 metrics parity. The same reasoning_tokens / costUsd discipline applies in the DDX RAG sidecar; the pattern is consistent across services.
  • SSE observability is operational, not financial. Don't conflate "publish failures" with "cost." Different teams, same service module file.
  • GDPR / data residency. Usage records contain userId, sessionId, modelSlug, token counts, and cost — no prompt content. Safe to retain longer than ChatMessage; the audit trail survives even after message redaction.

Related Topics

  • [[tucan-assistant]] — dispatcher writes the L4/L5 results that feed consumption.
  • [[ai-chat-sessions]] — owner of ChatMessage + TucanUsageRecord schema.
  • [[tucan-orchestrator]] — L5 persistency layer that writes the rows in the persist-then-publish order.
  • [[tucan-dispatcher]] — emits the per-stage cost via tool result envelopes that the orchestrator persists.
  • [[sse-event-engine]] (W1) — SSE observability service measures publisher health.
  • docs/claude-context/CLAUDE_DATABASE.md — full Prisma schema reference.
    TUCAN Consumption Tracking and Pricing — Dudoxx Docs | Dudoxx