prototyping-testing
Prototyping and testing skill set covering prototyping strategy selection, usability test planning, heuristic evaluation, A/B experiment design, tree testing, five-second tests, concept testing, and r
SKILL.mdplugins/loxihart-design/skills/prototyping-testing/SKILL.md
GitHubPrototyping + Testing
Choose the right method, run it correctly, interpret results accurately.
<intake>Route by what the user needs to do:
| User Says | Route To |
|---|---|
| "how should I prototype this" / "what fidelity" / "paper vs clickable" | references/prototyping-strategy.md |
| "usability test plan" / "write task scenarios" / "test plan" | references/usability-testing.md |
| "evaluate this design" / "find usability issues" / "heuristic review" | references/heuristic-evaluation.md |
| "A/B test" / "experiment design" / "test this variation" | references/ab-experiment-design.md |
| "tree test" / "validate navigation" / "IA testing" | references/tree-testing.md |
| "five-second test" / "first impressions" / "does this land" | references/five-second-concept-testing.md |
| "concept test" / "validate this idea" / "assumption mapping" | references/five-second-concept-testing.md |
| "measure design quality" / "HEART framework" / "SUS score" / "SEQ" | references/research-metrics.md |
If unclear: Ask — "Are you choosing how to test, planning a test session, or evaluating an existing design?"
</intake> <method_overview>| Method | What it answers | When to use |
|---|---|---|
| Prototyping strategy | What should I build to test this? | Design is not yet testable |
| Usability testing | Can users complete tasks with this design? | Have a prototype or working product |
| Heuristic evaluation | Where are the usability problems? | Need expert review without recruiting users |
| A/B experiment | Which version performs better in production? | Have traffic and a metric to optimize |
| Tree testing | Can users find things in this navigation? | Validating or diagnosing an IA |
| Five-second test | Does the first impression communicate correctly? | Landing pages, new features, brand validation |
| Concept testing | Is this the right direction before we build? | Pre-design or early concept stage |
| Research metrics | How do we measure design quality over time? | Defining what good looks like with numbers |
Before running any test session:
- Research question defined — what decision will this inform?
- Success criteria written before the session (not after)
- Participant screener matches the actual user population
- Tasks use scenario framing with no feature names in wording
- Metrics defined: what you will measure and how
- Recording consent obtained
- Analysis plan written before data collection begins
- Stakeholder readout scheduled before sessions start
| Reference | Purpose | When to Read |
|---|---|---|
references/prototyping-strategy.md |
Fidelity types, decision matrix, build timeline | Choosing prototype approach |
references/usability-testing.md |
Test plan components, task writing, moderator protocol, analysis | Planning or running a usability test |
references/heuristic-evaluation.md |
10 Nielsen heuristics, severity scale, evaluation process | Conducting a heuristic review |
references/ab-experiment-design.md |
Hypothesis format, sample size, anti-patterns, go/no-go criteria | Designing an A/B experiment |
references/tree-testing.md |
Setup, metrics, directness vs first-click, benchmarks | Validating information architecture |
references/five-second-concept-testing.md |
Five-second test process, desirability testing, assumption mapping | First impressions and concept validation |
references/research-metrics.md |
HEART framework, SUS grading scale, SEQ benchmarks | Measuring and interpreting design quality |
CLAIMS
- [SOURCED] Nielsen: 5 participants for qualitative moderated usability testing catch ~85% of usability problems on average — nngroup.com/articles/why-you-only-need-to-test-with-5-users. Caveat: assumes L=31%. For well-optimized interfaces (L < 31%), more users needed. Faulkner (2003) found groups of 5 found as few as 55%. Treat 5 as minimum for qualitative iteration, not a guaranteed ceiling.
- [SOURCED] SUS average is 68 (50th percentile, C grade on Sauro-Lewis curved grading scale); ≥80 is the common industry goal — Sauro & Lewis (2016); grading scale first published Sauro (2011) measuringu.com/sus
- [UNVERIFIED] Tree testing scale attributed to Albert & Tullis, Measuring the User Experience (2013): <60% = significant structural problems; 61–80% = good; 80–90% = very good; >90% = excellent. Cannot confirm this exact breakdown from the primary source — treat as [GENERATED] until verified against the book.
- [SOURCED] Optimal Workshop Treejack: 80%+ per-task success rate = well-performing IA — support.optimalworkshop.com/en/articles/2626846
- [SOURCED] Practitioner consensus (treetesting.atlassian.net): 65+ overall score is acceptable; below 65% = significant IA problems warranting investigation
- [SOURCED] MeasuringU (Sauro, 2013): mean tree test completion 66% across 77 tasks/3 studies — below usability test averages because search and visual cues are absent — measuringu.com/tree-testing-ia
- [SOURCED] [PERSON] (Neo Insight): usability scores average 29% higher than tree test scores on the same IA — a 70% tree score may correspond to ~90% in-product findability
- ! "Below 60% requires redesign" is practitioner shorthand — "warrants significant IA investigation" is the accurate framing; "redesign" is an inference, not a sourced claim
- [SOURCED] SEQ is more sensitive than SUS for individual task-level measurement because it captures perception immediately after a specific task, not the overall system
GAPS / BLINDSPOTS / UNKNOWN-UNKNOWNS
- Remote unmoderated testing tools (Maze, Lyssna, UserTesting) have different reliability characteristics — sample bias is a significant concern for non-panel recruitment
- Accessibility-specific testing methods (screen reader sessions, switch access testing) are not covered
- Statistical power calculations for A/B tests assume normally distributed data — non-normal distributions (revenue, session length) require different approaches