Model Cost / Rate / Budget Governance

How NanaSelect keeps AI-model spend bounded, observable, and resilient (issue #1191, part of capability #1185). Real models cost money and rate-limit, so every model-calling pass — corpus embedding (#1187), media embedding (#1189), and the media visual-analysis pass once it calls a real model (#1188) — runs under one governance layer. It sits alongside the model bindings in RAG Intelligence Bindings and the egress control in Media Egress Governance.

The one-line version: a per-run cap and a hard monthly budget bound every model call, transient errors retry with backoff and then degrade to the deterministic path, and every run's call/item volume + estimated spend is recorded and shown in the admin analytics view — all tunable via declared config, no code change.

Posture (decision #1195)

Property	Value
Monthly budget	$50/mo hard cap on estimated model spend (MODEL_BUDGET_MONTHLY_USD)
Per-run cap	2000 items per pass run (MODEL_MAX_PER_RUN); over-cap work defers to next run
Config	Declared env vars — tunable without a code change (AC-04)
Retry	Bounded exponential backoff on transient / rate-limit errors, then degrade (AC-03)
Observability	Append-only spend ledger + admin "Model spend" panel, per model & pass (AC-02)
Spend figure	Estimated — Workers-AI bills per neuron; call + item volume are exact

The budget cap (AC-01, AC-04)

resolveModelBudget(env) (src/lib/server/model-budget.ts) reads the two env vars into a { monthlyUsd, maxPerRun }, defaulting to $50 / 2000. Each pass computes its effective per-run cap as:

spentThisMonth = monthlySpendUsd(db)                 // sum of the ledger, this month
budgetCap      = effectiveRunCap(model, spentThisMonth, budget)
             = min(maxPerRun, floor(remaining$ / perItem$))
maxEmbeds      = min(caller's maxEmbeds, budgetCap)  // the tighter of the two

So the cap shrinks as the month's estimated spend approaches the ceiling and reaches 0 when the budget is spent out — all remaining work simply defers to a later run (the pass reports deferred), extending the --max deferral pattern the stub passes already used. Budgets are declared config: change the env vars, no code change.

Retry & degrade (AC-03)

Every model network call in a pass is wrapped in withModelRetry:

• Transient / rate-limit errors (429, 5xx, timeout, network) retry up to attempts (default 3) with exponential backoff (baseDelayMs · 2^n).
• Non-retryable errors (a 400, a bad request) throw immediately — retrying wastes budget.
• On exhaustion the last error is rethrown, so the pass aborts to its partial result rather than hanging. The retrieval read paths (corpus-retrieval, similarAssets) independently degrade to the deterministic path (lexical / brute-force / stub) when a model arm is unavailable, so the app never hangs on a model outage (ties to the reversibility work #1193).

Spend observability (AC-02)

Each pass run appends one row to the model_spend_log ledger (src/lib/server/analytics/model-spend.ts): pass, model, calls, items, estimated_usd_micros, created_at. It is append-only and carries no visitor PII.

• Where it's shown: the Model spend panel on /admin/analytics — this month's estimate per (model, pass) and the total against the monthly cap.
• Reads: monthlySpendUsd(db) (the budget-guard total) and modelSpendSummary(db) (the per-model/pass rollup) both window on the current month via monthStart.
• Estimate, not billing: estimateSpendMicros(model, items) applies a conservative per-1000-items cost (MODEL_COST_PER_1K_USD, tunable in code). Workers-AI bills per neuron, so the dollar figure is for watching the trend toward the cap, never a billing source of truth; call and item counts are exact.

Configuration

Env var	Default	Effect
MODEL_BUDGET_MONTHLY_USD	50	Hard monthly ceiling on estimated model spend.
MODEL_MAX_PER_RUN	2000	Max items one pass run processes before deferring the rest.

Declared in wrangler.jsonc vars; a missing/non-positive value falls back to the default. Tune either without a code change.

Where it applies

Pass	Module	Governed calls
Corpus embedding (#1187)	corpus-embedding-pass.ts	npm run corpus:embed, ingest
Media embedding (#1189)	media-embedding-pass.ts	npm run media:embed
Visual analysis (#1188)	media-analysis-pass.ts (caller)	admin "Run analysis pass"

Each call site resolves the budget from env and passes it through; the two embedding passes additionally wrap their embed call in withModelRetry and record spend. The visual-analysis pass inherits the config-driven per-run cap now and records real spend once #1188 wires a real (egress-gated, #1190) analyzer.