creating-uat
Build a complete User Acceptance Test for a codebase — an exhaustive markdown checklist written to the tester's skill level (novice / intermediate / expert), plus an HTML form the tester fills in (pas
SKILL.mdskills/creating-uat/SKILL.md
GitHubCreating a UAT
Produce three things for the target codebase:
UAT-<app>-<YYYY-MM-DD>.md— an exhaustive manual acceptance checklist covering every user-facing surface, written so someone who has never programmed can execute every step.UAT-<app>-<YYYY-MM-DD>.html— a form generated from that markdown where the tester records Pass / Fail / Not done, a "what happened" note, a look-and-feel comment with severity, and screenshots (uploaded, dragged, or pasted with Ctrl+V).- A helper program the tester runs. Save keeps their work; Submit validates the run and writes
every answer, comment and screenshot link back into the markdown file, plus a
results.json.
The completed markdown is then the input to the processing-uat skill, which turns failures and comments into a fix plan.
Core rule: make no assumptions. Ask before you write (Phase 0). A wrong assumption produces a UAT that tests the wrong app.
Phase 0 — Orient, then ask (REQUIRED)
Spend ten minutes reading the repo before you ask anything. Good questions are downstream of
discovery. Asked cold, a question about integrations comes out as "do you have any third-party
integrations?" — which makes the user do your reading for you. Asked after a look at the code, it
becomes "there is an OAuth router but no GOOGLE_CLIENT_ID in .env.example — is Google actually
configured?" Only the second is worth a turn, and only the second uncovers anything.
The questions below are the required minimum, not the whole list: add one whenever the code raises something the answer would change — an integration that may not be wired up, a feature that looks half-built, a destructive operation you would need permission to test.
Read enough to make the options concrete — no more:
README.mdand anyCLAUDE.md/AGENTS.md- the task runner (
package.jsonscripts,Makefile,pyproject.toml) anddocker-compose*.yml - the names in
.env.example— names only, never values - the feature/route/module directories, to see what actually exists
Then use AskUserQuestion with concrete options. Skip a question only if the user already answered it.
Scope — the whole app from a clean machine, or only part of it? Default and recommended: everything, including install and setup.
Starting state — what does the tester already have? (nothing at all / dependencies installed / services already running / an account already created). This decides whether setup steps are in.
Environment — which operating system and shell will the tester use? Get this right before writing a single command; Windows PowerShell and macOS bash need different text.
Destructive tests — include irreversible procedures (data deletion, restore-from-backup, key rotation, migration rollback)? Offer "include, clearly flagged" vs "exclude".
Tester skill level — ask this explicitly, with these three options. It decides how much detail every step carries, and it is the single biggest factor in whether the checklist is usable:
- Novice — has never opened a terminal. Every physical action spelled out, full glossary, "how to open a terminal" section. Recommend this when in doubt.
- Intermediate — comfortable with a computer and can run commands, but does not know this codebase. One logical action per step, project vocabulary only.
- Expert — a developer or QA engineer. Assertions rather than scenery, verification through API/logs/database where that is faster, no hand-holding.
Ask who will physically execute the checklist if the user is unsure, and write for the lower level when two people of different levels will share one run. Read
references/detail-levels.mdfor the full standard and a worked example of the same test at all three levels.Credentials and test data — what accounts, keys, or seed data will exist? A test that needs a login the tester does not have is a dead test.
If an answer leaves a real ambiguity, ask one targeted follow-up. Do not invent an answer.
Phase 1 — Discover every surface (read the code, never guess)
Map the live surface area from the repository every time. Fan out with parallel Explore agents
when the codebase is large. Read the orienting documents first: README.md, CLAUDE.md,
package.json / pyproject.toml / Cargo.toml scripts, docker-compose.yml, .env.example, and
any roadmap or status document that records what is actually built.
Cover all of these that exist:
| Surface | What to extract |
|---|---|
| Setup / operations | every prerequisite, install command, container, migration, seed, env var (name + what it does), and every script in the project's task runner |
| UI | every page/route, every button/link/form/modal by its real on-screen label, and each state: empty, loading, success, error |
| UI addresses | the URL of every page you write a test for — these become uat:url markers so the tester can open the screen and annotate it |
| HTTP API | every endpoint: method, path, auth tier, a copy-pasteable request, success status, error statuses |
| CLI | every command and flag, with exact syntax and expected output |
| Background work | cron jobs, queues, workers, webhooks, scheduled tasks — how to trigger one and how to see it ran |
| Data | what the user can create, edit, delete; what persists across a restart |
| Unhappy paths | wrong input, bad/expired credentials, missing records, permission boundaries, duplicate submissions |
| Security | secrets absent from served HTML and API responses, authorization actually enforced, destructive actions gated |
Only test what is built. Cross-check every candidate against the repo's own status notes. Anything
deferred, parked, or stubbed goes into a ## Not in scope (deferred — do NOT test) section so the
tester knows it was excluded on purpose, never into a test that is doomed to fail.
Capture exact strings: commands, routes, button labels, env var names, file paths. A beginner cannot fill in a blank.
Stop if there is nothing to test
If discovery finds no runnable surface — no server, no page, no command, no endpoint; only configuration, documentation, or a specification for something not yet written — stop and say so. Do not generate a checklist.
This matters because the failure is invisible: a UAT written from a README's promises or a
CLAUDE.md's conventions reads perfectly plausibly, and the tester only discovers it is fiction when
every test fails. Report what you found instead (a spec, an install of someone else's tool, an empty
scaffold), name what would have to exist before a UAT is meaningful, and ask whether a different
repository was intended.
The same applies in the small: a feature that exists only as a mock-up with hardcoded data is not built. Either exclude it, or test it honestly — see Testing a known gap below.
Phase 2 — Propose where the files go
Suggest, then confirm before writing:
<repo>/docs/uat/UAT-<app-slug>-<YYYY-MM-DD>.md the checklist
<repo>/docs/uat/UAT-<app-slug>-<YYYY-MM-DD>.html the form (generated)
<repo>/docs/uat/assets/<same-slug>/ screenshots the tester attaches
<repo>/docs/uat/tools/ the helper scripts, copied from this skill
docs/uat/ is the default because the results belong in the repo where a fix plan can cite them. If
the project has no docs/ directory, offer uat/ at the root instead. Use today's date. Use a short
app slug, lowercase, hyphenated.
Also add to the project's .gitignore (ask first):
docs/uat/.uat-progress-*.json
Phase 3 — Write the markdown
Read references/uat-file-format.md before writing — the marker syntax is a contract the scripts
depend on. references/example-uat.md is a working example of the whole shape.
Every test needs an <!-- uat:test id=N.M --> marker under its ### N.M Title heading, inside a
section that has a <!-- uat:section id=N title="..." --> marker. You do not need to write the
answer blocks — the generator injects them.
Record the level from Phase 0 in the meta block as tester_level: novice | intermediate | expert, so
the file states who it was written for and the form can show it.
Set base_url: in the meta block to where the app runs (http://localhost:3000), and give every test
about a specific screen a <!-- uat:url /that/path --> marker directly under its uat:test marker.
The form then offers Open this screen and annotate it on those tests, and whatever the tester pins
comes back into that test's notes and screenshots. Tests without the marker are unaffected — use it
wherever a test is really about one screen, and leave it off for setup, CLI and API tests.
The shape of every test
Every test has the same five parts at every level — Goal, Before you start, Steps, PASS looks like, If it does not work. Only the amount of detail inside them changes.
### 3.2 Save a new report
<!-- uat:test id=3.2 -->
- **Goal (plain words):** Store a report so it is still there tomorrow.
- **Before you start:** Test 3.1 is done and passed.
- **Steps:**
1. Click the blue **New report** button in the top-right corner.
2. Click the box labelled **Title** and type `My first report`
3. Click the blue **Save** button at the bottom of the form.
- **PASS looks like:** The form closes and `My first report` appears at the top of the list on the
left, with today's date next to it.
- **If it does not work:** Write down any red text that appears, and attach a picture of the screen.
How much detail — read references/detail-levels.md
That file is the standard for the level chosen in Phase 0, with the same test written three ways. The short version:
| Novice | Intermediate | Expert | |
|---|---|---|---|
| Steps | one physical action each | one logical action each | one outcome each |
| Glossary | every technical word | project vocabulary only | none |
| Section 0 | how to open a terminal | prerequisites with versions | one line |
| Expected result | what appears on screen | the observable outcome | an assertion |
| Verify via | the UI only | the UI, plus simple commands | whatever is fastest — API, logs, database |
Coverage does not change with level. Every surface from Phase 1 gets a test whoever is running it; only the wording changes. An Expert checklist is shorter, not thinner.
Writing rules — apply at every level
- Give the exact text to type, in backticks, complete. Never
npm run <something>. If a value must be filled in, show a clearly marked placeholder and say where to get the real value. - Name what to click using the real label discovered in Phase 1.
- Write down the expected result. The tester verifies by comparing; a step with no stated outcome cannot be passed or failed.
- Binary PASS/FAIL. If two people could reasonably record different results from the same observation, the test is written badly.
- One verifiable claim per test. Merging three assertions into one checkbox makes a failure impossible to triage.
- Order by dependency, and by side effect. Sequence install → configure → start → use → operational/destructive, and state each test's preconditions in Before you start. Then check the other direction: does any test sabotage the ones after it? A rate-limit test that locks sign-in for fifteen minutes, a test that deletes the record a later test reads, a teardown placed mid-checklist — each strands the tester. Move it later, or warn in bold what it will cost them.
- Flag every destructive step in bold, with what is lost and how to undo it.
- Say what to capture on failure, pitched at the level: the exact red text for a Novice, the failing response and log line for an Expert.
- Keep the tester unblocked: if one test fails, say whether they can carry on.
- Branch on the unexpected. "If you do not see X, do Y first." (Essential for Novice, useful at every level.)
- Never ask the tester to write down a secret. The completed checklist gets committed, mailed
around and pasted into chat. Ask for the name of a key, never its value: "list the variable
names in
.env" rather than "paste your.env". When a test needs a credential, tell the tester where to get it and have them confirm it works — never have them record it. Say so explicitly in any test that goes near one, and in the security tests add: if a secret is exposed, report which name appeared and nothing more.
Do NOT add "how did this look?" questions to the tests — the form gives every test its own look-and-feel comment box with a severity, and lets the tester report anything the checklist never asked about.
Testing a known gap
Sometimes the honest test is one that passes by confirming a defect is still there: a page that is a mock-up with fixed data, a feature reachable only by typing a URL, a limitation the team has accepted for now. Leaving these out lets the next person mistake the mock-up for a working feature; writing them as ordinary tests produces a failure nobody intends to fix.
Write them so the tester cannot misread which way is which:
- State the gap in the title — "Tasks are a demo and do not save", not "Test the Tasks page".
- Say in the Goal that confirming the limitation is the point, so a tester does not record Fail out of sympathy for the app.
- Make PASS the current behaviour: "PASS looks like: the tick is gone after a reload — the change was not saved, confirming this page is still a demo."
- Make FAIL mean the gap has closed: "If your change survives the reload, tasks now persist and this test needs rewriting. Say so."
Use this only for gaps you have confirmed in the code. A guess dressed up this way is worse than no test at all.
Coverage requirement
Complete means every surface from Phase 1 has at least one test, including setup and teardown, every
UI state, every endpoint/command/job, the unhappy paths, and the security checks. Group by surface,
number N.M. Err on the side of more tests — but not past the point where the run stops happening.
A checklist nobody finishes is worth less than a shorter one they complete. So:
- Estimate the run and put it at the top, in the "what you need" section: roughly a minute per Intermediate test, more for anything with a wait in it. Sixty tests is about an hour; a tester who budgets thirty minutes will abandon it at test 25 and you will get a partial run.
- Split by session, not by trimming coverage, when it runs long: put the setup and the highest-risk features in the first checklist, operational and edge cases in a second. Two files that both get completed beat one that does not.
- Never merge tests to shorten the list. One verifiable claim per test stays true at any length — a merged test cannot be triaged when it fails.
Shell gotchas to bake in
- Windows PowerShell: use
curl.exe, never barecurl(it is an alias forInvoke-WebRequestand ignores-X/-H/-d). Never inline a JSON body with\"escaping — write it to a file with single-quotedSet-Contentand send-d "@body.json". GuardCopy-Item .env.example .envbehind aTest-Pathcheck so it cannot silently overwrite a real.env. - Long-running commands (a server, a watcher): tell the tester to open a second terminal and
leave the first running, and how to stop it (
Ctrl+C). - Dry-run the setup commands in the real shell where you can, and correct anything that does not work.
Phase 4 — Generate the form
Copy this skill's scripts/ directory (including scripts/assets/) into <repo>/docs/uat/tools/ so
the UAT folder is self-contained and works on a tester's machine that has never had this skill
installed. Then generate:
# Python (no dependencies, needs Python 3.8+)
python docs/uat/tools/generate_uat_html.py docs/uat/UAT-myapp-2026-08-23.md
# or Node (no dependencies, needs Node 18+)
node docs/uat/tools/generate-uat-html.mjs docs/uat/UAT-myapp-2026-08-23.md
Pick the runtime the target repo already uses. Both produce byte-identical output.
The generator normalises the markdown in place (injecting answer, summary and findings blocks) and reports the section and test counts. If it prints a file problem — duplicate id, a test outside a section, an unclosed answer block — fix the markdown and run it again.
Verify before handing over:
- The reported test count matches the number of tests you wrote.
- Open the HTML and read it as the beginner would: can every step be done with zero outside knowledge?
- Nothing deferred is presented as testable.
Phase 5 — Hand it to the tester
Give them exactly this, filled in with the real path:
- Open a terminal in the project folder.
- Type this and press Enter:
python docs/uat/tools/uat_server.py docs/uat/UAT-myapp-2026-08-23.md(ornode docs/uat/tools/uat-server.mjs docs/uat/UAT-myapp-2026-08-23.md)- Your web browser opens the checklist. Leave that terminal window open while you work.
- Work top to bottom. Every test starts at Not answered yet — change it to Pass, Fail or Not done as you go, and write what you saw. Use the purple box for anything you do not like the look or feel of, even when the test passed. Add pictures by clicking the picture box, dragging a file in, or pressing Ctrl+V.
- Where a test offers Open this screen and annotate it, click it. The app opens in a new tab. Try the thing the test asks for, then click the Annotate this page button on your bookmarks bar (the checklist shows you how to put it there, once). Click anything on the screen to drop a numbered pin, say what is wrong, then press Done. Your notes and a picture go straight onto that test — switch back to the checklist tab and they are already there.
- Click Save my work whenever you like — it also saves itself every minute.
- When everything is answered click Submit. If you cannot finish, use Submit what I have so far — the report is then clearly marked as unfinished.
- Press
Ctrl+Cin the terminal when you are done.
Tell them the Submit button refuses an incomplete run and lists exactly what is missing with a "take me there" link, so they cannot accidentally hand in half a UAT.
If the tester opens the .html file directly instead of running the helper, the page still works and
keeps answers in their browser, but shows a banner and switches Submit off — it cannot write to a file
without the helper.
Phase 6 — After the tester submits
Submit rewrites the markdown: a summary block at the top (counts, failed IDs, IDs with concerns), each
answer filled in under its test, and a findings section for anything the tester reported that no test
asked about. It also writes <name>.results.json for tooling.
Point the user at the processing-uat skill to turn that file into a fix plan.
Reference
references/uat-file-format.md— the markdown contract: markers, answer blocks, summary, findings, sidecar files, validation rules. Read this before writing a UAT.references/detail-levels.md— the Novice / Intermediate / Expert writing standards, with the same test written at all three. Read this after Phase 0, before writing any test.references/example-uat.md— a small complete example, written at Novice level.scripts/generate_uat_html.py,scripts/generate-uat-html.mjs— markdown → HTML form.scripts/uat_server.py,scripts/uat-server.mjs— save / upload / submit helper.scripts/assets/uat.css,scripts/assets/uat.js— the form itself, inlined at generation time.scripts/assets/annotate.js— the pin-and-comment overlay that runs on the app under test. The checklist embeds it in the bookmarklet, so a strict Content-Security-Policy cannot block it.scripts/assets/vendor/html2canvas.min.js— turns the annotated page into an image (MIT; seevendor/.vendor-note.md). It re-renders the DOM rather than photographing the screen, which is why the tester is shown the result and asked whether it looks right before it is sent.
Troubleshooting
| Symptom | Cause and fix |
|---|---|
no sections found |
a ## heading is missing its <!-- uat:section id=N title="..." --> marker |
test X is not inside any section |
the test marker appears before the first section marker |
answer block id=X does not belong to the test above it |
an answer block was moved or its id edited — put it back under its test |
duplicate test id |
two tests share an id; ids must be unique |
| Port 8777 busy | the helper walks up to 8796 automatically; the URL it prints is the one to use |
| Screenshots do not show in the markdown | they are relative links — open the .md from inside docs/uat/, and keep the assets/ folder next to it |