Deploy & Activate Intelligence (RAG)

What/why: Intelligence rollout — flags, defaults & reversibility — the flag inventory, degrade guarantees, and reversibility this runbook operates.

Purpose: turn on the "intelligence" features — hybrid corpus retrieval (#1187), visual media tags (#1188), media similarity (#1189) — in production as a documented, verifiable sequence rather than tribal knowledge, with a one-flag rollback for each.

When to use: activating any intelligence swap in prod for the first time, or re-verifying the deployed bindings after a deploy.

The conservative gate (read first)

Every swap ships OFF by default (decision #1195 — defaultOn: false for all three in src/lib/server/intelligence-flags.ts). A swap flips ON only after BOTH:

1. its #1192 eval reports GO (candidate clears the metric bar and the lift bar — see quality-eval-harness.md), and
2. Paul approves the flip.

Until then the request-time path stays deterministic (lexical corpus scoring, stub media analyzer, brute-force D1 cosine). Never flip a swap on a NO-GO, and never flip more than one swap per verification pass — you want a clean before/after.

Where to see the current state

/admin/ai → "Intelligence flags" (#1581) renders the live inventory — one row per swap, fed by resolveIntelligenceFlagStates (the same resolvers that gate the code, so it can never drift). Columns: Swap (which feature, with its originating issue), Flag key (the env var or app_settings key), Live state (ON/OFF right now), Default (always off, #1195), Backing (env — deploy to flip vs app_settings — instant flip), and Degrade path (what serves when the flag is OFF or the model arm fails). The surface is read-only — flips are made only through the procedures below, never from the admin screen.

Prerequisites (one-time)

• Cloudflare auth: wrangler login (or CLOUDFLARE_API_TOKEN) for the paul@iwpi.com account.
• Vectorize indices exist (768-dim, cosine — the dimension is immutable):

  wrangler vectorize create nanaselect-corpus-text --dimensions=768 --metric=cosine   # corpus (#1186)
  wrangler vectorize create nanaselect-media-sim   --dimensions=768 --metric=cosine   # media  (#1189 / ops #1218)

• R2 buckets exist:

  wrangler r2 bucket create nanaselect-corpus
  wrangler r2 bucket create nanaselect-media

Activation sequence

Run in order. Steps 1–5 are safe (no swap is live yet); step 7 is the only behaviour change, and it is gated.

1. Apply remote D1 migrations

npm run db:apply:remote        # wrangler d1 migrations apply DB --remote

2. Confirm the Vectorize indices resolve + round-trip

The prod bindings smoke checks (#1226) embed → upsert → query → delete a throwaway vector against the deployed account, proving AI + each index actually work. Vectorize has no local emulation, so these hit the remote indices in every mode.

npm run rag:smoke        # AI + VECTORIZE_CORPUS
npm run media:smoke      # AI + VECTORIZE_MEDIA

Both must print ✅ … PASSED (exit 0). A failure means the index is missing or a binding is misconfigured — fix before proceeding (a missing media index prints the wrangler vectorize create hint).

3. Deploy the worker

npm run build && wrangler deploy      # see docs/getting-started/deployment.md

4. Populate the real corpus + media

Feed real content, then ingest + embed it into production (#1224 remote path, #1225 feed):

# Corpus: capture real nanawall.com content → files → prod D1/R2 → prod vectors
npm run drupal:load -- extract                   # dump → corpus (single path, #1369)
npm run corpus:ingest -- --remote --yes          # → prod D1 + R2 CORPUS
npm run corpus:embed  -- --remote --yes          # → VECTORIZE_CORPUS

# Media: embed referenced assets → prod vectors + provenance
npm run media:embed   -- --remote --yes          # → VECTORIZE_MEDIA + D1 media_embeddings

--remote prints the production target and requires --yes; runs are idempotent and #1191-budget-bounded (an interrupted run resumes cleanly). See corpus-refresh.md and corpus-ingestion.md.

5. Run the #1192 evals and review GO/NO-GO

npm run corpus:eval      # recall@8 lexical → hybrid  (corpus-retrieval / #1187)
npm run sim:eval         # precision@12                (media-similarity / #1189)
npm run tag:eval         # f1                          (visual-tags / #1188)

Read the verdict line, not the exit code. Each eval prints a GO — … / NO-GO — … line but always exits 0 — the GO/NO-GO is a human signal. (The only exit-code-enforced eval is the CI regression guard, npm run test:unit -- src/lib/server/eval.) The bars are in DEFAULT_EVAL_BARS and documented in quality-eval-harness.md.

Evals tab (#1823) — the persisted evidence surface. The admin console at /admin/ai?tab=evals triggers the visual-tags golden-set eval in-console (candidate parameterized by a named profile, run through the same egress gate as production), persists every run (eval_runs: profile snapshot, metrics, verdict vs bar), lists the history, and compares two runs side by side. This is where the GO/NO-GO evidence for the review below lives — running an eval never flips a flag; activation remains the manual flow on this page.

6. GO/NO-GO review + approval

Bring the eval verdicts to Paul. Proceed to step 7 only for swaps that report GO and are approved. A NO-GO swap stays off — investigate (more/better corpus, tuning) and re-run the eval.

7. Flip the flag (with approval)

One swap at a time:

Swap	Flag	How to flip ON	Takes effect
Hybrid corpus retrieval (#1187)	CORPUS_HYBRID_RETRIEVAL	add "CORPUS_HYBRID_RETRIEVAL": "on" to wrangler.jsonc → vars, then wrangler deploy	on deploy
Media similarity (#1189)	MEDIA_SIM_VECTORIZE	add "MEDIA_SIM_VECTORIZE": "on" to vars, then wrangler deploy	on deploy
Visual media tags (#1188) + AI descriptions (#1816)	media.egress_enabled	toggle ON in /admin/privacy (D1 app_settings)	instant — no deploy

The two env flags are absent from wrangler.jsonc vars today, so they resolve OFF — flipping ON means adding the key. Companion tuning vars (CORPUS_HYBRID_W_LEXICAL / CORPUS_HYBRID_W_VECTOR / CORPUS_HYBRID_TOPK, MEDIA_SIM_TOPK) are optional; defaults apply when absent.

8. Verify

Re-run the smoke checks (step 2) and spot-check the live behaviour (a corpus query now blends vector scores; "assets like this" now uses Vectorize; media analysis now runs the real vision analyzer and describer — an on-demand run on an image asset proposes tags and a description/alt-text from the image content, #1816). If anything looks wrong, roll back (below).

Reversibility / rollback

Every swap is reversible by flipping its one flag OFF — the request-time path falls back to the deterministic behaviour immediately and completely, with no data corruption (the vectors/rows simply stop being consulted). This is the #1193 guarantee; the full degrade-seam map is in intelligence-rollout.md.

Swap	Flip OFF	Falls back to
CORPUS_HYBRID_RETRIEVAL	remove/off in vars → wrangler deploy	lexical corpus scoring
MEDIA_SIM_VECTORIZE	remove/off in vars → wrangler deploy	brute-force D1 cosine
media.egress_enabled	toggle OFF in /admin/privacy	stub analyzer + stub describer (#1123/#1816), no egress — instant

• The two env-flag rollbacks take effect on the next deploy; the media.egress_enabled rollback is instant (no deploy), the fastest kill switch.
• Rolling back does not require undoing the corpus/media population — the data is inert while the flag is off, and turning the flag back on reuses it.

References

• Intelligence rollout — flags, defaults & reversibility (#1193)
• Quality eval harness — go/no-go bars (#1192)
• RAG intelligence bindings — topology & provisioning (#1186)
• Media egress posture (#1190) · Model budget governance (#1191)
• Corpus refresh · Corpus ingestion · Deployment guide