Experimentation

What the Experiments screen manages — and why the product's design bets are tested rather than asserted.

What it is

An experiment splits visitors between two or more variants — either persona-path (track) variants or presentation (theme + layout) variants — with one arm marked as the control, a goal metric from the behavioral metrics dictionary, and guardrails (minimum sample size per arm, minimum runtime). Assignment is sticky per visitor; when the guardrails are met, an admin promotes the winner as the new default.

Why it exists

The rationale behind NanaSelect is a set of falsifiable bets, not settled fact — guided paths should convert where filters abandon, right-sized recommendations should survive bids (why it works). The measurement plan defines how those bets are judged; experiments are the apparatus that judges the smaller, ongoing ones: which track wording, which presentation, which path actually moves the goal metric.

Building experimentation into the admin (rather than bolting on an external tool) means the things worth testing — tracks, themes, layouts — are already first-class entities, assignment happens server-side where consent and bot-filtering already live, and promotion applies the winner with no deploy. Guardrails exist so a run can't be concluded on noise.

In the admin

Screen	How-to
Experiments	Managing experiments

Learn more

• Measurement plan — the hypotheses and metrics this apparatus serves.
• Behavioral metrics dictionary — the goal metrics an experiment can target.
• Presentation experiments walkthrough.