nickstire-verify
Use before committing or pushing changes to the nickstire app (apps/nickstire/) — the real local verification sequence, the brand-voice linter false-positive trap, and the prerender regen rule.
SKILL.md.claude/skills/nickstire-verify/SKILL.md
GitHubnickstire-verify
Verify a nickstire change is safe to commit/push. The master gate is
pnpm run verify, but it has a few non-obvious traps that have burned
multiple sessions.
Run, in order (from apps/nickstire/)
bash scripts/preflight.sh(BASE=<GitHub main sha>from a hook-free clone): the CI gates that turned PRs red, about 4 minutes. Every push bills a full CI run (2026-10-10: 14 PRs, 29 pushes). Run it asnode scripts/agent-os/heavy.mjs run -- bash apps/nickstire/scripts/preflight.shFROM GIT BASH, sobashresolves and the heavy lease is kept (from PowerShellbashis not on PATH and the run fails with "'bash' is not recognized"); expect it to wait while another session holds the heavy lease or the chrome slot. It catches the proc-census, knip orphan-gate and fail-open-slice reds that a lane's own test files miss (witnessed 2026-10-18: 3 real reds). A test that reads a DATA file at runtime (readFileSync) is selected only by the text scan, never byvitest related(witnessed #3152: preflight green, CI red oncompletionAuthority.test.ts): after anydocs/operations/*.jsonedit runnode scripts/preflight-text-scan.mjs <file>and the tests it names.pnpm run check—tsc --noEmit. Must be 0 errors.pnpm test— vitest. Must endTest Files N passed (N), exit 0.pnpm run build—vite build+ esbuild server + maybe-prerender.- (full gate)
pnpm run verify— env + check + lint + source-lint + hooks-lint + route-validate + tests + build.
Before pushing a diff that touches client/ — the DoD compiler
The completion-authority steps of the agent-policy job derive their requirements per-diff. Touching any
client/ file (an edit, a rename, even a DELETION) derives an
operator-walkthrough requirement, and any evidence already on main was
written for a PREVIOUS diff — so it is stale by definition and fails the
gate.
Write the evidence for THIS diff in a NEW per-PR fragment,
.completion/evidence.d/<branch-slug>.json (format in that directory's
README): { "evidence": { "operator-walkthrough": { "ref": "..." } } }, or
{ "deferred": "<reason>" }. Say what the operator sees, and if the answer is
"nothing", prove it (zero importers, identical render) rather than asserting
it. A new file per PR never conflicts with a sibling PR. Freshness is by
content: an entry whose exact text is already on the merge-base (any fragment,
any legacy key) reads STALE, so copying old evidence does not pass. Rewriting
the legacy .completion/evidence.json entry still counts, but conflicts with
every concurrent PR — don't.
Witnessed on #1428: red in 22s on a stale entry, green in 31s once rewritten.
One check name, FOUR different sub-gates — a red is not "the same gate again." Witnessed on #1830: the completion-authority steps of the agent-policy job went red four times on one PR, each a different sub-gate, ~50 minutes of fix-push-poll cycles because only sub-gate 1 was documented:
- Stale or missing per-diff completion evidence (above) — fix: add a
fragment in
.completion/evidence.d/written for THIS diff. - Unresolved P1 review threads from the Codex review bot — fix: reply,
then resolve via GraphQL
resolveReviewThread. - A capability-ledger cross-axis rule (
operator_onlyrequires >=integration_verified) — fix: obey the ledger ladder; fix the LABEL, never inflate the state to satisfy the rule. - An un-rendered
REALITY-LEDGER.mddiff — fix: always runscripts/render-reality-ledger.mjsand commit the.mdbeside the.json.
Traps
- A new tRPC read that calls
invokeLLMmust be listed inclient/src/lib/unbatchedProcedures.tsand pass a boundedslotWaitMs. Otherwise one slow call freezes every batched sibling panel on the page (witnessed PR 3137,lot.officeDigest). - Never text-match a database error. drizzle-orm 0.45 wraps every driver
error; the wrapper's message is the SQL and its params, and the code/errno
live on
.cause. A regex onerr.messagemisses the real 1054/1062/1146 and matches any query whose params contain those digits or whose SQL names the table. Twenty sites did this until #2574/#2589. Useserver/lib/dbErrors.ts; test with drizzle's ownDrizzleQueryError, not a look-alike, plus a wrapped timeout whose params hold the code as the control. - A test within ~2x of its timeout is a shuffled-order failure waiting to
happen.
tableWriterCoveragepassed at 13s in default order and timed out at 34-36s under--sequence.shuffle.files --sequence.seed=29, reproducibly (2026-09-23; fixed by reading each file once, 0.14s). Sweep a few seeds before shipping test changes, and compare runs only on the SAME file set: a branch with a different test-file count shuffles into a different order, so "main passes seed 29" proved nothing until the same tree ran without the change. .completion/evidence.jsonconflicted on nearly every base merge (witnessed 3x on 2026-09-23) because sibling sessions rewrote the same per-diff keys. Fixed by per-PR fragments (above) — write a fragment, not the manifest. If an OLDER branch still carries a manifest edit and conflicts: resolve three-way, per key, against the merge-base — take the side that changed; if both changed, OURS stays current and THEIRS is kept as<key>-superseded-<date>-sibling-main; write withjson.dumps(d, indent=2, ensure_ascii=False) + "\n"(round-trips byte-exact); then re-runnode scripts/dod-compiler.mjs --base origin/main --enforce. A merged PR's conflict also stopspull_requestCI from running at all: "no checks" on a pushed head means checkmergeable_statefirst.- Wait on a background suite by its SUMMARY line, never
pgrep -f. A loop likeuntil ! pgrep -f "vitest run"matches its own command line (which contains the pattern) and never exits; two such waiters sat out a 600s timeout on 2026-09-23. Poll the log for^\s+Testsinstead, or capture$!. - A derived scan set must include the WIRING file it derives from.
server/__tests__/cronNoSwallowedFailure.test.tsscanned every modulescheduler.tsimports and neverscheduler.tsitself — which held 13 inlinecatch { return { details: "X failed" } }wrappers, one of them re-swallowing a rethrow the same wave had added one frame below. The "every cron fails loudly" claim was false for 13 jobs while its canary was green (2026-09-02, found by an independent reviewer). When a gate scans "every module X imports", scan X too, with its own positive control: the inline handler inside the registry is the shape a module scan cannot see. - The SMS
uncertainoutcome means do-not-retry, not not-delivered.sendSmsreturnssuccess:true, uncertain:trueon a shop-gateway timeout andserver/sms.tsdocuments it as "the relay may well have delivered". Any change to how a send result is interpreted must be checked against the FOUR outcomes inserver/lib/smsOutcome.ts: a claim is consumed for every outcome except a definite failure (smsClaimConsumed); only the COUNTERS keep uncertain apart from sent; never collapse uncertain tofailedon a receipt a human reads — it invites a re-send. The 2026-09-01 wave's first rule ("consume only on sent|queued") re-texted customers every tick at five sites until the second review caught it. - A multi-commit wave is not done after the author's own pass. Over a
10-commit wave the author's hostile re-read found 5 defects; three
parallel independent reviewers (silent-failure hunter on
server/, contract reviewer on schema + cross-app, client/docs reviewer) over the full diff found 24 more, including two P0 regressions the wave itself introduced and three corrections to its audit claims. Before calling a wave done, run a hostile per-subsystem pass over the full diff, in-session unless the operator asked for agents (he rejected two unrequested Explore spawns on 2026-10-08), with a file:line checklist and a "verified OK / not checked" answer shape, then verify the top findings yourself. A single author pass over more than ~20 files has not once been sufficient (2026-08-12 onward). - A numeric test fixture is a claim about the PRODUCER's scale or
domain. Before inventing a value (0.05 vs 5.0, cents vs dollars,
x10000 vs percent), grep the function that produces it in production
and one existing consumer that renders it — a self-consistent
wrong-scale fixture keeps every test green around a real x100 bug.
Witnessed 2026-08-13: the real producer (
getTopPosts(),server/pipelines/instagram-data.ts:576) returnsengagementRateon a PERCENTAGE scale (5.23 for 5.23%), but two test fixtures invented a 0-1 fraction production never produces — every test stayed green while a real reel would have rendered "523.00%" (#1558, caught only by a post-merge audit, fixed in #1561). as never/as unknown as Xon a JSON.parse'd field silences the exact compiler check that would catch a nonexistent field or a wrong-domain value. Validate through a normalizer that degrades unknowns honestly, and readreel_jobs.payloadthroughshared/reelJobPayload.ts'sparseReelJobPayload()— never a fresh ad-hoc inline type. Witnessed 2026-08-13, two instances in one diff:attentionMicrostructureStore.tscastbrief.ctaType as neverfor a field no real payload has (makinghasCtastructurally always false), anddailyReelPost.ts:493force-cast raw JSON intoEntailmentVerdictthe same way — both merged in #1558 behind a green suite.- An e2e "proof" test that hand-builds the input a real IO function
normally derives can bypass the exact gate it claims to prove. If
the derivation is inline in the IO layer, extract it into a pure
shared function and call THAT from both the IO layer and the test.
Witnessed 2026-08-13:
scanfinishRun2EndToEnd.test.tshand-fed all 14 Local Discovery topics unfiltered, silently skipping the e_check -> government_feed evidence-gate split thatgatherTopicSignals()performs inline — the docstring claimed "the REAL functions" while bypassing the one load-bearing derivation (#1558); fixed in #1561 by extractingsplitLocalDiscoveryTopics()so the IO layer and the test share one implementation. - Brand-voice linter on CSS class names. The pre-commit
brand-voicelinter regex-matches banned words (premium,tier, etc.) in the staged diff — including CSS class names. The project-wide Tailwind utilitybtn-premiumappears in 5+ existing files but triggers the lint when added in a new diff. The lint message itself suggestsgit commit --no-verify— DO NOT. The correct fix is to edit the newly-added content: rephrase comments and removebtn-premium(or any flagged utility) from newly-added classNames. Pre-existing usages stay untouched (the linter only checks the staged diff). - The exit code lies in two different ways — read the SUMMARY LINE.
pnpm test | tailreportstail's status (0), not vitest's. Andpnpm test; echo "EXIT=$?"— which this skill used to recommend — is no better underrun_in_background: a compound command reports the LAST command's status, so the completion notification said "exit code 0" while 24 files and 234 tests had failed (2026-08-08). Run the command ALONE and readTest Files … passed/failed. If no summary line was printed at all, the run did not finish — see the teardown-crash trap below. The same holds for EVERY gate, not only vitest: never pipe a gate's output.node merge-gate.mjs ... | tail -4; echo EXIT=$?printedtail's exit, andpnpm run check | tail -3read as "tsc exit=0" while tsc had failed (2026-10-18, twice in one session). Redirect to a file, then read the file and the exit code separately (cmd > log 2>&1; echo $?).merge-gate.mjsneeds the FULL 40-character sha: a short sha prints usage and exits 64. pnpm run verifycan hit a pnpm-bootstrap glitch. Ifverifyfails before any of the inner gates run (no actionable output), fall back to running each gate individually (check / test / build).- The full suite can CRASH AT TEARDOWN and print no summary. On
Windows the default pool has ended with
ELIFECYCLE Test failedafter every test passed — observed 2026-08-08 with 528 pass markers, 0 fail markers, 0 "Failed Tests" banners, and no summary line (a libuvUV_HANDLE_CLOSINGassertion appeared from a separate script the same session). Do not read that as a regression, and do not read it as green. Triage by counting markers —grep -cE '^ *(×|❯) 'returning 0 means nothing failed — then re-run under--pool=forks --poolOptions.forks.singleFork=truefor an actual summary. Marker counting is triage; the summary line is the receipt. - Never mutate the working tree while a suite is running. A
background suite reads the tree continuously, so a
git checkout,git stash, or edit mid-run rewrites files under it and produces phantom failures. The tell is that failures cluster in ONE vitest project — 2026-08-08: 24 failed files, every oneclient/**, zeroserver/**, and a clean re-run passed 425/425. Wait for the run, or branch before starting it. - When a fail-open is safe only because another middleware runs
first, that ORDER is the security control — pin it with a test.
adminProcedurechainsrequireAdminIdentity(throws FORBIDDEN for non-admins) beforerequireFreshMfaAndPermission(which contains a deliberate, documented fail-open to pre-RBAC owner when the role row is unreadable). The fail-open is correct — choosingviewerwould let a DB hiccup lock the owner out mid-shift, the exact lockout reversed on 2026-07-16 — but it is safe ONLY because authentication already ran. As of 2026-08-08 no test covers that ordering, so a refactor reordering.use()calls would silently turn an authorization fail-open into an authentication one. Assert that the EARLIER gate rejects, not just that the later one behaves. - Migrations are hand-applied SQL. After editing
drizzle/schema.ts, generate / hand-write thedrizzle/NNNN_*.sqlmigration and apply it to the DB before re-runningpnpm run check. There is no auto-migrate. The check fails until the migration is applied. - Prerender is generated, not hand-edited. Changes to nav, SEO,
NAP, or page copy that should appear in
prerendered/*.htmlneedpnpm run prerender(orPRERENDER_ON_BUILD=1 pnpm run build). CI audits this. The SPA hydrates on top of prerendered HTML — so a nav change reaches users via the JS bundle immediately, but the prerendered HTML (what crawlers see) stays stale until regen. - A check that a page EXISTS is not a check that it CARRIES its
content. Prod serves two documents for nickstire routes: crawlers
get ~150KB of committed prerendered HTML, browsers get the ~14KB SPA
shell. Assert the payload (row counts, card counts, absence of a
not-found branch), not just the metadata. Witnessed 2026-08-15:
prerender:semantic-checkvalidated title/description/canonical/H1/ NAP/JSON-LD across 9 routes and passed all of: a 1-star review under the homepage's "five-star reviews" headline, a price-comparison route with ZERO per-size floor rows while the live endpoint served 9, and 10 blog articles serving HTTP 200 with "ARTICLE NOT FOUND" whilecontent.articleBySlughad full content for every one — ten acquisition pages invisible to search for four days, all green (#1588). - Never run
pnpm run regenlocally to refresh the committed prerendered tree.regen(scripts/regen-prerender.mjs) rewrites the tracked tree — a different script fromprerenderabove, which only writesdist/. A LOCAL regen loses DB-backed payloads; a CI regen loses Places-API-backed payloads unlessGOOGLE_MAPS_API_KEYis set in that workflow. Check what the environment can actually reach BEFORE regenerating, and diff the payload afterwards. Witnessed 2026-08-15: a local regen (6d99b9e3c) broke all 10 blog articles plus the price floors; the CI refresh that then "fixed" it regressed/reviews(132,918 bytes / 5 cards -> 107,213 bytes / 0 cards) because that workflow has noGOOGLE_MAPS_API_KEY— the tree has oscillated between environments for weeks (#1588). - After every regen, verify the swap didn't silently drop routes.
pnpm run regencan exit 0 and report "Broken: 0" while tolerating up to 10% failed routes — and the swap DELETES those routes' previous files. (1) grep the regen log for✗and forskipped. (2) Runnode scripts/check-prerender.mjsand git-restore any missing route dirs (git checkout -- prerendered/<route>) before committing. Witnessed 2026-08-19: exit 0 / "Broken: 0" while 10 blog routes failed on a latent scoping bug and the swap dropped their files (340 -> 327); two more routes timed out on the 50s budget the second run and also needed git-restore (#1709). lint:sourcematchesconfirm (/prompt (even inside a single-line JSX comment. Its comment-skip heuristic tests for a line starting with//,*or/*, and a{/* … */}line starts with{— so the words "confirm (two-tap)" inside a JSX comment failed a commit (2026-08-12,ApprovalsSection.tsx). Reword the comment; do not fight the regex and never--no-verify.lint:piifalse-positives have a sanctioned line waiver. A regex CONSTANT like/email/ireads to the linter as PII in a URL path (2026-08-12,services/activityLedger.ts— the matcher that MASKS emails). The mechanism is// pii-allow: <reason>on the line, and the reason is required. Readscripts/lint-pii.mjsbefore assuming a hit is real; it also allowlists the shop's own public numbers.- Do not fake timers around a sub-2 s REAL sleep that sits behind
await import().advanceTimersByTimeAsynccan run before the dynamic imports reach thesetTimeout, so the sleep never fires; local speed hides it.server/postInvoiceFollowUp.test.tspassed locally and timed out at 30 s in CI (2026-10-02, fixed infaa36e64). Use real timers with an explicit per-test timeout, or inject the sleep. - Run the fail-open-slice gate when a diff adds a test that slices source
text:
pnpm exec vitest run server/failOpenSliceGate.test.ts(3 ms), and usesliceBlock()fromserver/testUtils/sourceBlock.ts, never rawslice(indexOf(...)). Red in CI on 2026-10-01 (#2865), 2026-10-02 and again 2026-10-08 (tierCadence.test.ts,declaredSourceAtGeneration.test.ts); the full suite is the only other place it shows. - Do not
exporta value only so a test can read it. knip's orphan gate does not count a test as a consumer (ASS_CAPTION_MARGIN_V,DUE_CHECK_SLACK_MS, 2026-10-08: found after a 474 s suite, twice). Derive the value in the test from the function's behaviour or its output. When the diff adds anexport, runpnpm run lint:orphansin the targeted loop. - Before changing a call site, find the source-shape pins on it:
grep -rn '<fnName>(' --include='*.test.ts*'. A test that asserts source text goes red only at full-suite time (contentExperimentArmKey.test.ts, 2026-10-08). - A render or pixel test uses the PRODUCTION asset (font, overlay, LUT). OS
faces are fallbacks only:
captionSafeZone.test.tsrendered with DejaVu Sans Bold as "wider, so conservative" and measured 936 px against an 894 px budget on every Linux box, while production's Anton measures 600 px. - A script that rewrites a committed data file aborts on a failed round-trip
check, it does not print. A Python edit of
angle-bank.jsonprintedround-trip identical: False, wrote anyway, and turned a 7-line change into a 1,515-line diff.sys.exiton mismatch; a file with its own layout gets a line edit. - A refusal on a path a scheduler calls must throw what the caller's catch
types.
dailyReelPostadvances the rotation only onReelPreflightBlockedError; four bareErrorthrows inenqueueReelJobwould have cost every following day's Reel (2026-10-08, one was a new gate whose test pinned the bare throw).reelEnqueueRefusalAdvance.test.tsnow fails on any new one; a test that pins the throw's text pins the bug. pnpm run lint:hooksaudits hook-after-early-return. A hook called after anif (...) return nullsilently breaks React. The lint catches it; don't bypass.pnpm run validate:routesconfirms registered routes match handler files. Run after adding / removing routes.
Instrument runs (cage match / ghost replay / any measurement script)
Three instrument-integrity failures in one arc (2026-08-06/07) share a shape: the measurement lied while every gate stayed green.
- A liveness probe must exercise the EXACT model/lane/config the
measured work uses. The 2026-08-07 cage gauntlet probed the ambient
default lane (gpt-4o, key present), then burned all 8 seeds against the
pinned adversary lane (Ollama, key absent). A probe of a different lane
is a false-green generator. Fixed structurally in #1416
(
probeBothLanes()); hold new instruments to the same bar. - A run that completed ZERO units of work must exit non-zero. The
same gauntlet printed
0 losseswith exit 0 after 0 matches ran. Zero-measured must never render as zero-findings. - Model lanes must be pinned in the instrument, never ambient.
Without
AI_FORCE_OLLAMAa bareinvokeLLMresolves toLLM_MODEL || gpt-4oon OpenAI; with it, EVERY request — explicit pins included — flattens onto one model, silently defeating duel role-diversity. Pin by Ollama-native substring name (routes with no flag); seeAGENT_MODELinscripts/cage-match.tsandGHOST_AGENT_MODELinserver/services/ghostReplay.ts.
Running a script that needs real credentials
- nickstire's
.envdoes NOT carry the Ollama key.OLLAMA_API_KEYlives inapps/statenour/.envlocally and on Railway in prod — grep proof 2026-08-07: absent from both the primary and worktree nickstire.env. Inject it per-shell for local instrument runs. - Background /
run_in_backgroundshells inherit NO inline env. Earlier runs worked only because the interactive shell happened to carry the key; the fresh background shell had nothing. Set the env in the SAME command that runs the script. - Scripts that need a foreign-app key must fail fast naming its home
(pattern:
scripts/cage-match.tsafter #1416).
Multi-session push protocol
main is shared between concurrent Claude sessions (e.g., nickstire
- statenour). On a push:
git fetch originfirst.- Read
git log origin/main..HEAD --onelineto see all unpushed commits — including the other session's. Whoever pushes carries everyone's committed work. - The pre-push hook runs
turbo run build --filter=...[$UPSTREAM]for both apps. If the other session has a broken build in its working tree, your push is also blocked. Do NOT--no-verify(that's the gate stopping a broken deploy). - Don't rewrite shared history (disrupts the other session).
- If blocked by the other app, surface to the operator with a self-contained prompt for that session to fix and push.
Deploy verification
After a successful push:
curl -s https://nickstire.org/api/healthreturnsuptimein seconds since the container started. A successful redeploy resets it. Poll uptime; when it drops to a low value, the new container is serving.- Browser caches OLD JS bundles aggressively. Verify a deploy by
either (a)
curlfor the actual bundle and grep for new strings, or (b) hard-reload with?cb=freshquery param to force a clean load. - Railway watch-paths are per-app:
apps/nickstire/**triggers a nickstire redeploy only;apps/statenour/**triggers statenour. A test-only file underapps/nickstire/still triggers a redeploy (bundle is identical, but container restarts).
Verifying a user-facing copy change
Source-grep is necessary and not sufficient. Both halves failed once on 2026-09-03 (fixed in #2099):
- Sweep case-insensitively. A grep for
"new ownership"missed the capitalized"New ownership"in a hero intro and it shipped. Usegrep -ri, always. - Confirm the rendered page in REAL Chrome, not the in-app browser. The in-app browser blocks the site's fonts, so it is not a fair visual check — the miss above was only caught by loading the live page in real Chrome.
- Source and served HTML can disagree. #2094's indexing was silently
defeated because 10 of 12 routes served STALE prerender snapshots with
the old
noindex. The drift-catcherclient/src/__tests__/prerender-indexability-consistency.test.ts(#2098) now proves theindexedflag,routes.ts,SITEMAP_ROUTESand the prerender snapshot robots all agree — run it after any indexability or copy change that a crawler reads.
Business-truth claims are operator-owned facts. shared/business.ts
is canonical and client/src/__tests__/canonical-business-truth.test.ts
enforces agreement with it. Never assert tenure, ownership, review count
or rating from code inference — Nick's is the SAME owner who renamed
Moe's Tire & Auto (owner-confirmed 2026-09-03), so "new ownership" and
"run by Moe" are both false.