NanaSage Chat Transport

The request path for NanaSage (issue #763 transport, #764 RAG composition): a Cloudflare Worker endpoint that accepts a chat turn, grounds it against the corpus, runs the grounded prompt through a server-side LLM adapter under a bounded timeout, and returns the reply — streamed or as a single body — under an explicit typed-error contract with carried-through citations. This is the foundation every other NanaSage requirement builds on (citations #765, grounding guard #766, refusal/scope guard #767, advisor hand-off #768, retention #769, chat UI #770). It is designed to fail plainly (retry / hand-off) rather than hang silently, and to ground, not guess.

• Route: src/routes/api/nanasage/chat/+server.ts
• Transport core: src/lib/server/nanasage/chat.ts (pure, unit-tested)
• RAG composition: src/lib/server/nanasage/rag.ts (pure + thin over #66, unit-tested)
• Buyer surface: src/lib/nanasage/NanaSageChat.svelte (the chat panel), nanasage-client.ts (SSE client), citations.ts (citation view-model) — reached from the results payoff in src/routes/select/SelectionFlow.svelte

Endpoint

POST /api/nanasage/chat[?stream=0]

Request body

{
	"message": "How wide can an SL45 folding wall open?", // required, ≤ 4000 chars
	"threadId": "…", // optional — defaults to the ns_session identity
	"context": {
		// optional; grounding passages are composed SERVER-SIDE from the corpus (#764) —
		// a client-supplied `passages` is ignored. Only `history` is honoured (#770).
		"history": [{ "role": "user", "content": "…" }]
	}
}

The thread id is the anonymous session identity (ns_session, session-store), resolved by the server hook — needsSession covers /api/nanasage. A client may pass its own threadId; when omitted the session id is used. NanaSage does not mint a parallel session concept.

Response — streaming (default)

Unless ?stream=0 is set (or the client does not accept text/event-stream), the reply is delivered as Server-Sent Events so a waiting/typing state can render past the ~1s threshold (median question→answer target ~3s, per #759). Each SSE data: line is one JSON event:

data: {"type":"delta","text":"An SL45 "}
data: {"type":"delta","text":"folding wall "}
data: {"type":"done","reply":{"text":"…","model":"stub-chat-v1","grounded":false}}

The stream always terminates with exactly one done or one error event. A mid-stream failure is delivered in-band as a typed error event (bytes are already flowing at HTTP 200), never as a broken connection.

Response — single body (?stream=0)

{
	"reply": { "text": "…", "model": "stub-chat-v1", "grounded": true },
	"citations": [
		{
			"key": "89b17184-…:0", // itemId:seq — matches the grounded context passage id
			"text": "…",
			"heading": "Summary",
			"provenance": {
				"itemId": "…",
				"title": "St. Francis High School",
				"contentType": "marketing",
				"sourceUrl": "https://www.nanawall.com/…",
				"version": "…",
				"fetchedAt": "…"
			}
		}
	]
}

Grounding (RAG composition, #764)

Every answer is grounded in retrieved corpus material — the transport never generates from the model alone. Before the model runs, the route composes a grounded turn (composeGroundedTurn, rag.ts):

1. Selection context from the session. The buyer's systems-in-view (recommended + alternatives + viewed) and guided-flow answers are read from the anonymous session (session-store) keyed by the thread id, so the buyer never re-explains context each turn.
2. Retrieval through the #66 interface only. Retrieval goes exclusively through retrieveBySelectionContext (flow answers), one retrieveBySystem per system in view, and the question arm retrieveByQuery on the buyer's message text (corpus-retrieval.ts, #66/#1407) — no parallel/bespoke corpus query. Results are merged and deduped by itemId:seq (highest score wins), ranked, and capped; the question arm holds reserved slots (QUESTION_ARM_RESERVE, 2 of 6) because system-arm passages score on curated vocabulary and would otherwise crowd out the very material that answers the asked question. The question arm runs only when selection context exists — a session-less turn still short-circuits to hand-off.
3. Grounded prompt. Retrieved passages are assembled into the prompt sent through the transport; the model answers only from that material.
4. Citations carried through. Each retrieved passage's full #66 provenance is returned as a citations entry (keyed to the grounded context passage id) so citation rendering (#765) needs no second lookup. In the stream, a citations event is emitted first.

Journey context envelope (recommendation-aware answers, #1426)

When the anonymous session carries a recommendation, every turn is composed with a journey context envelope (src/lib/server/nanasage/journey.ts) — the difference between a generic FAQ bot and an expert who knows what was just recommended:

• Contents (PII-free by construction): journey stage, the recommended system (id + name), up to 3 cheaper-sufficient alternatives, and up to 8 answered factors as question: choice labels. Lead data has no path in; the size caps keep retrieved passages the bulk of the model context.
• Stage comes from a validated client page hint (journey.page in the chat request body: selecting / results / comparing / browsing; anything else collapses to browsing). The chat component derives it from the URL, so the sitewide launcher (#1424) degrades gracefully on pages with no journey state.
• Injection: the envelope renders as a compact Buyer context: block before the reference material in the model request; the system prompt instructs the model to resolve deictic questions ("this system", "it") to the recommended system and to relate answers to the recommendation ("While the recommended SL73 …, it does offer …") — while still drawing every product claim from the reference material only.
• Guard interaction: the envelope text joins the #766 attribution set (buyer state is attributable truth, not a product claim), so recommendation-referencing sentences pass the lexical guard; it is not a citation and never renders as one.
• Degradation (unchanged-path guarantee): no session, no recommendation, or a lookup failure yields no envelope — the request is byte-identical to the pre-#1426 composition (proven by test).
• Measured, not vibes: the versioned eval set data/eval/nanasage-context-eval.json (#1192 pattern) carries the QA-motivating SL73 wood-finish case plus deictic/alternatives/control cases, scored by contextMentionScore (src/lib/server/eval/context-cases.ts).

Transcript retention (consent-gated, #769)

Chat transcripts are user content, so they follow the Iubenda/GTM-governed 2-tier consent + retention model (session-store, #347). A grounded turn's {user, assistant} pair is persisted to nanasage_transcripts only when the measurement tier (Tier 1) permits — the same tier that gates the anonymous selection_sessions snapshot. The gate is the single-source-of-truth retentionPermitted(consent) (transcript.ts).

• Consent is re-read per request (Iubenda cookie via readRequestConsent), never cached — a withdrawal is honoured on the very next turn.
• A non-consented session is in-memory only — nothing is written, so no record exists after it ends.
• Retention rides the session lifecycle: nanasage_transcripts.session_id → selection_sessions.id ON DELETE CASCADE, so the existing session TTL purge (#116) removes transcripts on the same schedule — no separate purge path.
• The write is background (waitUntil), so retention never delays the reply. Only substantive grounded turns are retained; refusals / hand-offs persist nothing.

Advisor hand-off (#768)

Any chat turn can escalate to the human Technical Advisor in one action — a persistent "Talk to a NanaWall advisor" control in the panel, and the resolution the refusal / "I don't know" states point to. A buyer-requested hand-off is always honoured, never deflected back into the bot.

POST /api/nanasage/handoff (nanasage-client.ts → requestAdvisorHandoff) takes the chat transcript, derives advisor-facing context from it (transcriptContext — the buyer's questions + a one-line summary, so the buyer never re-explains), resolves the buyer's system-in-view from the session, and returns the same WarmHandoffPayload the guided flow uses (handoff.ts, #456/#9) — now with the transcript attached. It reuses the existing warm hand-off (buildWarmHandoffForSystem funnels through the shared assembler); it does not build a second hand-off. With no session / no recommendation it degrades to a dealer-repfinder + transcript payload — the escalation still lands.

Grounded talking points (#1545). The payload carries talkingPoints: CitedPassage[] — up to 2 cited why-this-fits prose passages for the anchored system, from a second small retrieveBySystem (no content-type filter, lexical-only — hybrid pinned off per the #1542 cost posture) shaped with the pdp deep-link seam. The advisor panel renders them beneath the transcript summary, so an advisor opens with NanaWall's own words rather than resource links alone. Empty on miss or with nothing recommended — the #456 degrade contract; the panel renders exactly as before.

Scope + refusal guard (#767)

Before any retrieval or model call, the inbound message is screened for adversarial intent (assessScope, scope-guard.ts) — defense-in-depth that holds regardless of the model, since it matches the user's message, not the model's output:

• Prompt injection (injection) — attempts to override the assistant's instructions.
• Prompt exfiltration (prompt_leak) — attempts to leak the system prompt.
• Impersonation (impersonation) — attempts to make it pretend to be a human or the human Technical Advisor (#9).

A refused message never reaches the model. It returns a graceful hand-off whose reply discloses that NanaSage is an AI and offers the advisor path — reason set to the scope kind (injection / prompt_leak / impersonation). Genuinely off-topic questions are not handled here; they pass and are declined by the grounding guard (#766).

The guard is conservative to avoid over-blocking: "does the cero ignore the sill?", "show me the panel options", and "are you human?" all pass (the last is answered honestly, not refused — only a request to impersonate is). The exported SCOPE_PROBES set (scope-guard.ts) is a documented, repeatable regression artifact — scope-guard.spec.ts asserts every probe is refused as its kind, so a regression fails the build.

Anti-hallucination guard (#766)

Grounding a turn's retrieval is not enough — the generated answer itself is checked before any byte is displayed (assessGrounding, grounding.ts). Every substantive sentence must be attributable to a retrieved passage: it is attributed when it either contains a verbatim quoted phrase from a passage (a direct quote is grounded even when framing dilutes its coverage) or enough of its significant tokens appear in the passage set. Framing/connective sentences (below the claim-length floor) are exempt.

An answer with a sentence that no passage supports is blocked and routed to the same first-class refusal/hand-off as zero-retrieval — reason ungrounded rather than no_grounding. The guard checks the output text, not the model, so it holds when a real model swaps in (#1188). Because the whole answer must clear the guard before display, the streaming path completes-then-guards-then-segments rather than forwarding native model deltas.

The guard is pure and deterministic (grounding.spec.ts), adding no model call.

Zero-retrieval → hand-off (never an ungrounded answer)

When retrieval returns nothing (no selection context, empty corpus, all-stale), the route does not call the model. It returns a hand-off outcome so the UI routes to the refusal / advisor path (#767/#768):

// non-streaming (HTTP 200)
{
	"handoff": { "reason": "no_grounding" },
	"reply": {
		"text": "I don't have supporting NanaWall material… Let me connect you with a NanaWall advisor…",
		"grounded": false
	}
}

Streaming callers receive a single {"type":"handoff","handoff":{…},"reply":{…}} SSE event. A client-supplied context.passages never grounds an answer — only the corpus does; context.history is still honoured for multi-turn continuity.

Error contract

Every failure is a typed error the UI branches on — never a silent hang or opaque 500. retryable tells the UI whether the same turn may be retried or must be reworked / handed off.

kind	HTTP	retryable	Meaning
invalid	400	false	Malformed body, empty message, or bad thread id
timeout	408	true	The bounded timeout tripped
rate_limited	429	true	Upstream model rate limit
upstream	502	true	Any other upstream/model failure

Non-streaming shape: { "error": { "kind": "timeout", "message": "…", "retryable": true } }. Streaming shape: the same object inside a {"type":"error","error":{…}} SSE event.

Request-time spend gate (#1590)

Chat is the only endpoint that pays a model per request, so on top of the per-IP rate limiter (#1582) the route enforces a hard daily budget for the nanasage-chat pass — checked before any retrieval or model work (recordModelSpend only observes after the fact). The gate reads today's estimated spend from the model-spend ledger (dailySpendUsd, pass-scoped so offline batch passes never count against chat) and compares it to the env-first lever resolved by model-budget.ts:

• MODEL_BUDGET_DAILY_USD — explicit daily ceiling when set and positive;
• otherwise MODEL_BUDGET_MONTHLY_USD (default $50) pro-rated over 30 days.

An over-budget turn routes to the same first-class hand-off as the guards above — reason budget, HTTP 200, advisor path offered — never an opaque 500. The gate fails open: an unreadable ledger admits the turn (the rate limiter still bounds volume), mirroring the limiter's own fail-open contract. Specs: spend-gate.spec.ts (route), model-budget.spec.ts + model-spend.spec.ts (cores).

Model adapter seam

The LLM call goes entirely server-side through a pluggable ChatModel adapter — the key/binding is never exposed to the client. The seam mirrors the established convention (media-analysis.ts, decisions #1123/#1113): an interface with a readonly model provenance stamp and a deterministic stub shipped as the default.

interface ChatModel {
	readonly model: string;
	complete(turn: ChatTurn, opts: ChatModelOptions): Promise<string>;
	stream?(turn: ChatTurn, opts: ChatModelOptions): AsyncIterable<string>; // optional native streaming
}

• Production adapter: anthropicChatModel (anthropic-chat.ts, #1377) — the real Anthropic Messages API via @anthropic-ai/sdk, default model claude-opus-4-8. Selected by resolveChatModel when ANTHROPIC_API_KEY is bound and the configured model id is claude-*; with no key (or NANASAGE_MODEL=stub-chat-v1) it fails closed to the stub — a flag flip, no code change (#1193). See Real model (#1377) below.
• Fallback: stubChatModel — deterministic, no network, no key. It quotes the top grounding passage when present; ungrounded, it plainly says it has no source material (it never invents). It remains the unit-test model and the degrade target.
• Native vs segmented streaming: when an adapter implements stream() the transport forwards native deltas; otherwise it segments complete() so the streaming surface works regardless. The guard-first pipeline (#766) completes-then-segments either way.

Real model (#1377)

anthropic-chat.ts implements the seam against the Anthropic Messages API. The composition is the #764 contract expressed as the API shape: the NanaSage instruction as the system prompt, prior turns as alternating messages, and the numbered reference material + question as the final user message.

• The system prompt is deliberately EXTRACTIVE. The #766 guard attributes a sentence by significant-token overlap with the retrieved passages (or a verbatim 4-token phrase), so fluent paraphrase — however faithful — reads as ungrounded. The prompt instructs the model to reuse the material's own wording, quote specs/names/numbers verbatim, and add no closing commentary about what the material doesn't cover (an honest "the material does not provide ratings" sentence is itself unattributable and would void the whole reply). Measured effect (10-shot eval, grounded cold-climate turn): guard pass went from 0/10 (paraphrase-style prompt) to 10/10, and answers got shorter — model-call median fell from ~6.5s to ~3.2s (p90 ~4.5s; route-level question→cited-answer ~4.2–5.8s including retrieval + guard).
• No sampling params are sent — claude-opus-4-8 rejects temperature/top_p (400). The NANASAGE_TEMPERATURE knob only affects the stub; prompting is the steering mechanism. thinking is likewise omitted for latency.
• Typed, sanitized errors (AC-05). SDK failures are mapped to ModelCallError (rate_limited / upstream / invalid) with sanitized messages — no provider text, URL, or key material can reach the client. classifyModelError honours the pre-classified error; an abort (the transport timeout) bubbles untouched so timeout stays authoritative.
• Refusal sentinel (#1407). The prompt instructs the model to reply exactly "I don't have material that answers that." when nothing relevant is supported; the route detects it (isModelRefusal) before the guard and routes to the advisor hand-off. An honest refusal must never render as a cited answer — a short refusal sentence can clear the lexical guard by coincidental token overlap and would otherwise display with irrelevant sources attached.
• Grounded-or-degrade (withStubFallback). A retryable upstream/rate-limit failure degrades the turn to the stub (which still answers verbatim from the top passage and clears the guard) instead of erroring at the visitor; provenance (reply.model) truthfully reports whichever model produced the reply. Non-retryable config errors and timeouts surface through the typed contract — misconfiguration is seen, not masked.
• Spend is recorded. Each real (non-stub) reply logs one model_spend_log row (pass=nanasage-chat, background via waitUntil) through the #1191 governance path; claude-opus-4-8 is priced in model-budget.ts.

Configuration (Worker vars)

Model and params are configuration, never hardcoded at call sites (resolveChatConfig, mirroring resolveHybridConfig). Missing / out-of-range values degrade to conservative defaults, so misconfiguration yields a working transport.

Var	Default	Meaning
NANASAGE_MODEL	claude-opus-4-8	Model selection: claude-* → Anthropic adapter (needs ANTHROPIC_API_KEY), stub-chat-v1 → stub
NANASAGE_TEMPERATURE	0.3	Stub-only sampling knob — not sent to Anthropic models (rejected on Opus 4.8)
NANASAGE_MAX_TOKENS	512	Max reply tokens, [1, 8192]
NANASAGE_TIMEOUT_MS	8000	Bounded timeout, [250, 60000] ms
NANASAGE_PRESTREAM_TIMEOUT_MS	model timeout + 12000	Total pre-stream budget (#1923): every await before the first SSE byte. Clamped to ≥ model timeout + 12 s margin, ≤ 120 s; expiry → typed timeout (408, retryable), never a zero-byte hang

ANTHROPIC_API_KEY is a secret (.dev.vars locally, wrangler secret put in prod), never a var. Without it the transport fails closed to the stub.

Timeout & cancellation

runChatTurn / runChatTurnStream run the model under an AbortController raced against timeoutMs. On timeout the controller is aborted (so a real model can cancel its in-flight request) and a typed timeout error is returned. The abort signal is authoritative: because only the timeout timer aborts it, a model's own abort-rejection can never be mis-typed as upstream — it stays a timeout.

Buyer surface (chat UI, #770)

The chat surface lives in the guided selector (SelectionFlow.svelte), disclosed inline from the results payoff — a discoverable "Still have questions? Ask NanaSage" affordance that expands the panel without leaving the flow. The buyer's selection context is already stored server-side, so the chat grounds on their session automatically.

Sitewide launcher (#1424). NanaSage is additionally reachable from every page — a floating lower-right pill (NanaSageLauncher.svelte, mounted once in the root layout) opens the same chat in a native-<dialog> modal on all public routes and the admin area. The modal reuses the WelcomeModal accessibility bundle (top layer, focus trap, Escape, backdrop light-dismiss; focus returns to the launcher on close) and mounts the chat lazily on first open. Styling is --ns-* token-driven, so each public theme (and the admin theme) colors it; on phones the pill collapses to the AI badge.

One conversation per session (#1424). Every chat instance binds to a shared, sessionStorage-backed thread (chat-session.svelte.ts singleton over the pure chat-thread.ts codec — settled turns only, tolerant parse). Chat inline on results, open the modal anywhere: the conversation continues; it also survives an in-tab reload. Session identity / consent handling is unchanged (#769 retention still gates server-side transcript storage).

NanaSageChat.svelte renders:

• AI-identity disclosure at the top of the panel — the buyer knows they're talking to an AI that cites its sources, not a human (aligns with the no-impersonation scope guard, #767). The prose is a single flowing text block beside the AI badge (#1919 — bare text nodes in the flex row previously wrapped as disjoint columns).
• New chat (#1919) — a control above the thread (hidden while the thread is empty, disabled mid-send) that resets the shared #1424 conversation: empties the singleton's turns in place (clearTurns, so both the inline and launcher-modal surfaces reset), removes the sessionStorage key, and clears the surface's advisor-handoff state.
• A threaded, multi-turn conversation — user and assistant turns accumulate; each assistant turn keeps its citations visible; prior turns feed context.history for continuity.
• Incremental streaming with a waiting state — deltas render as they arrive; a pulsing typing indicator appears only once a response exceeds the ~1s threshold (#759), so a fast reply never flashes it. While a turn streams the send button becomes a Stop control (#1920): aborting settles the partial answer as a done turn (citations already delivered stay; an empty bubble is dropped — pure settleAbortedTurn) and re-enables the composer. The thread auto-scrolls to follow new tokens only while the buyer sits at the bottom (pure isPinnedToBottom decision) — scrolling up to read pauses the follow; the instant (non-animated) scroll is reduced-motion safe by construction.
• Starter questions (#1920) — an empty thread offers four corpus-plausible questions as one-tap chips; a tap fires a distinct nanasage_starter beacon and sends the question as the first turn.
• Markdown answers (#1920) — assistant turns render lightweight markdown (paragraphs, lists, bold/italic) via answer-markdown.ts: a hand-rolled, closed-alphabet renderer (p/ul/ol/li/strong/em/br only) that HTML-escapes ALL input before any transform, so model output can never inject markup; any parse failure degrades to escaped plain text. Streaming partials render incrementally (an unterminated **run stays literal).
• Per-answer feedback (#1920) — each settled assistant turn offers a changeable 👍/👎 (aria-pressed). Votes POST to /api/nanasage/feedback and upsert into the nanasage_feedback table keyed by (session_id, turn_hash) — the payload carries an FNV-1a hash of the answer text plus the journey page, never the text itself, so transcript storage stays consent-gated (#769). Endpoint failures are silent to the buyer (public surface degrades); the route reuses the generous session rate limiter.
• Inline citations (the #765 view-model) — a serif pull-quote with the source title, content-kind badge, v3 · Jan 2026 label, and a source link resolved by the shared #1422 rule ($lib/citation-links.ts, same as the recommendations page): URL-bearing sources (direct captures and Drupal view/dump ingests — docs, articles, PDFs) link to their canonical nanawall.com URL under the #1008 liveness gate; a pdp citation carries its system's captured PDP path (attached server-side by attachPdpPaths, matched deterministically against systems.pdp_path) and resolves through the $lib/deeplink seam (#414, #1425); anything unresolvable degrades to plain text — never a dead link. Links open in a new tab (rel="noopener") in both the inline and the launcher-modal contexts. A long extract (past the pure excerptCollapsible threshold — >240 chars or >3 explicit lines) renders clamped to ~3 lines with a per-citation, keyboard-accessible Read more / Show less toggle (aria-expanded); a short extract shows no toggle (#1919).
• Typed-error retry and a distinct hand-off rendering for a zero-retrieval turn.

The client (nanasage-client.ts) requests the SSE stream and normalizes every response (SSE or JSON) into one ChatWireEvent union (citations / delta / done / handoff / error), so the component branches on a single shape.

Testing

The pure cores — config resolution, turn validation, error classification, the timeout race, streaming, and stub determinism — are unit-tested in src/lib/server/nanasage/chat.spec.ts with no live binding (this repo's convention). The Anthropic adapter's pure halves — request composition, typed error mapping, model selection, and the stub fallback — are covered in anthropic-chat.spec.ts; the live call is exercised end-to-end against the dev server with the real key. The RAG composition (rag.spec.ts) and the client + citation view-model (src/lib/nanasage/nanasage.spec.ts) are likewise unit-tested. The chat surface's pure cores are covered in the same pattern: the thread codec, reset, abort-settle, and scroll-pin decisions in chat-thread.spec.ts; the markdown sanitize/degrade path in answer-markdown.spec.ts; feedback hashing/payload shaping and the silent-failure submit in feedback.spec.ts. The route and the chat panel are thin glue over those cores and are exercised end-to-end against the dev server.