Skip to content

Validation log — trial designs against the brief

Part of the design-brief package (issue #263, final AC): the brief is validated by generating trial designs from it and recording what it failed to communicate. Each trial gets an entry; gaps feed brief revisions.

Trial 001 — 2026-07-02

Setup. A fresh agent was given only BRIEF.md + assets/tokens.json (the exact hand-off payload) and asked for the §8 deliverables scoped to a trial: the 40-token theme, a stepper mock (Interior acoustic control), a results mock with the 645→640→SL45 nudge, and candid delivery notes. Output: trials/trial-001/ (theme.css, stepper.html, results.html, delivery-notes.md).

Outcome. The brief communicated the design intent well — the mocks got the register, the nudge-as-good-news, one-decision-per-screen, token-only styling, and real content right without access to the underlying docs. What it failed to communicate falls into four classes, recorded below.

Class A — brief errors (fixed in BRIEF.md immediately)

GapFix
§3.1 called success-soft a no-source gap, but tokens.json carries color.background.highlight (green-01 #f0f8e8) — a natural success-softGap list corrected
§7's breakpoints (480/768/1024/1280) silently disagree with tokens.json's breakpoint tokens (375/540/768/1024/1366/1600)§7 now states explicitly that it overrides the token file for breakpoints

Class B — brief omissions (next revision of the package)

  • Token-contract friction the brief must warn about: no radius-none (brand buttons are square — 0px — but radius literals fail the build); only one font-size token while banning size literals; brand radius-full = 50% breaks pills; brand shadow.md is modal-grade (rgba .5) and unusable on calm cards; focus == accent makes focus rings on teal buttons invisible without an offset gap; text-muted/text-subtle grays (#6e6e6e vs #707070) are indistinguishable, so a 4-step text ramp isn't buildable from extraction alone; no accessible text-faint source.
  • Unstated layout decisions: page ground (bg vs surface both resolve white; is the canvas white or warm?); long-education-vs-fold on mobile (never-collapse + 3 paragraphs pushes options below a phone fold on the P1 screen); progress a11y pattern (progressbar vs step list); keyboard semantics of click-again-to-clear.
  • Rule tension to resolve in the brief: "one primary action per screen" vs. "the nudge has its own action" on results (trial resolved by register — teal primary vs green success action — a good resolution worth codifying); nudge scope (single alternative vs. the whole 645→640→SL45 chain); "Compare these 3" referent.

Class C — data the brief can't carry (needs a data appendix or explicit

"ask, don't invent" scope)

The trial had to invent: factor option lists (names, counts, thresholds, "Not sure yet"), per-system attribute values (STC ratings — the substance of the nudge's "still satisfies" claim), tier assignments (even for the hero example: 645=Premium, 640=Mid, SL45=Budget are guesses), canonical factor order ("Step 6 of 14"), shortlist membership, system one-liners, and all education copy beyond the single STC-45 sentence. Revision decision: the hand-off should attach a small real-data appendix (the 14 factors with options, tier map, 2–3 real education narratives, sample attribute rows for the nudge example) — mocks built on invented specs can't validate H4 (real-content survival).

Class D — hand-off packaging gaps

No logo/wordmark asset, no font files (mocks fell back to system fonts), no sample photography, header/footer/nav unspecified, overlay untestable in P1 scope. Acceptable for mocks; note in the hand-off that brand assets ship separately when moving past mock stage.

Heuristic spot-scores (trial-001 mocks)

H1 nudge 5 · H2 one-decision 5 · H3 token fidelity 4 (proposals flagged properly; forced size literals are a Class-B brief defect, not a design defect) · H4 real-content 3 (real names, but invented specs — Class C) · H5/H6 not exercised in trial scope · H7 4 · H8 4 (focus-visibility workaround needed — Class B). The brief passes its own bar for what a brief controls; the failures it produced are informational gaps, now recorded.

Upstream notes

The token-contract frictions (Class B, first bullet) largely mirror the gap audit already filed as nanawall-design-tokens#163; radius-none, the focus/accent collision, and the indistinguishable-gray text ramp are new evidence to add there when the contract additions are designed.