Skip to content

Content Auto-Tagging / Enrichment Engine

The enrichment engine (issue #454) auto-tags currently-untagged ingested content — blog posts, projects, resources — to the shared facet vocabulary (#344), so that content becomes retrievable by the same facets the guided flow uses. It is the write side of facet-filtered retrieval; the read side is the corpus retrieval contract.

It lives in src/lib/server/content-enrichment.ts (the engine) and src/lib/server/admin/content-tags.ts (the review layer); the admin surface is /admin/content-tags.

The loop

propose (auto)  →  review (human: accept / reject / edit)  →  consume (retrieval)
  1. Propose — the enrichment pass scores each untagged, active corpus item against the facet vocabulary and writes proposed tags. Nothing is applied silently.
  2. Review — an admin accepts a proposal (it goes live), rejects it (it is remembered, never re-proposed), or edits it (corrects the facet, then accepts).
  3. Consume — accepted tags are read by the guided-flow retrieval arm, so tagged content surfaces for a matching selection context.

Data model

One table, content_facet_tags, holds both the proposals and the review outcome:

ColumnMeaning
item_idThe corpus item being classified (cascades on hard-delete of the item).
facet_kindattribute | factor — which shared-vocabulary set facet_ref indexes.
facet_refThe tagged facet's own id (an attributes.id or factors.id). Polymorphic — app-enforced, no FK.
statusproposed (awaiting review) | accepted (live, consumed by retrieval) | rejected (declined, not re-proposed).
sourceproposed (written by the pass) | human (an admin accepted/edited it).
scoreLexical relevance at proposal time.
proposed_for_hashThe item's content_hash when last (re)proposed — the incrementality key.

The shared facet vocabulary is attributes ∪ factors (#344); the tag is polymorphic over the two. The (item_id, facet_kind, facet_ref) unique index means a facet is proposed/decided for an item at most once.

The proposer

Matching reuses the corpus retrieval engine's deterministic lexical scoring (#66) — no embeddings, no LLM, no network, no budget. Each candidate facet carries signal terms: its own display name (weighted 3×) plus the taxonomy terms mapped to it in the shared vocabulary (1×). An item's extracted text is scored against every candidate; the top-scoring facets above a minimum score are proposed.

The scoring sits behind the ContentFacetProposer seam, so an embeddings/LLM proposer can be dropped in later without touching the persistence, review, or retrieval paths — the engine, the review layer, and the retrieval contract are all agnostic to how a proposal was produced.

The pure cores — proposeFacetsForText and planEnrichment — are exported and unit-tested (content-enrichment.spec.ts) without a live D1.

Idempotent + incremental

Running the pass (runEnrichmentPass, exposed as Run enrichment pass in the admin) is safe to repeat:

  • Idempotent — an item is skipped when it already has a tag proposed against its current content_hash. Re-running with no content change writes nothing.
  • Incremental — only new or changed items (by content_hash) are re-scored.
  • Never clobbers a human decisionaccepted and rejected tags are excluded from re-proposal, and only proposed rows are ever deleted. A facet no longer suggested for a re-scored item has its stale proposed row dropped; any human decision on it remains.

Review actions

The review layer (content-tags.ts) mirrors the media-tagging accept/reject/edit pattern (#376):

ActionEffect
acceptContentTagaccepted, human. Live in retrieval.
rejectContentTagrejected, human. Remembered; the pass won't re-propose it.
editContentTagCorrect the facet the tag points at, then accept (merges onto an existing tag for the same item+facet if present).
addContentTagA curator adds a tag the pass didn't propose (accepted, human).
listContentTagsThe review queue, resolved to facet + item display; filterable by status/item.

Writers validate the referenced facet (and item) exists before persisting — a tag is never written dangling. Every action and every pass run is recorded in the audit trail (#21) under the content-tag entity.

Consumption

Accepted tags are read by retrieveBySelectionContext — see corpus-retrieval.md § Facet-tag consumption for the contract. In short: the selection context is resolved to its facets, and items carrying an accepted tag for any of them are boosted so they surface through the same retrieval the guided flow already uses.