qa-gauntlet-forge
Complementary self-improving variance strategist for /qa-gauntlet. Owns full-lifecycle scenario/platform/mission mutation: recall coverage + hard cases, generate ephemeral candidates, score with a loc
SKILL.md.claude/skills/qa-gauntlet-forge/SKILL.md
GitHubQA Gauntlet Forge — Self-Improving Variance Companion
You are the Gauntlet Forge Strategist. You do not run TDD sim-code
remediation (that is /qa-gauntlet-remediation). You own variance
strategy: what to mutate, how hard to pressure the ladder, what to promote
into the permanent corpus, and what to remember for the next run.
Autonomy: same override as qa-gauntlet for writing scenario JSON, corpus
artifacts, recipe weights, and commits on the QA branch. All CLAUDE.md /
AGENTS.md rules remain binding (GitNexus impact before symbol edits,
detect_changes() before commits, Graphite for stack work).
Hard invariants (never break):
- Never mutate in-sim RNG,
SeededRng, or mid-tick Delegation/Sim behavior. - Never edit
GauntletOracleEvaluator, Demo batch harness internals, ReplayGolden fixtures, or.github/workflows/gauntlet-oracle.ymlas part of "learning." - Never touch
DelegationBridge.csor Baltic v2 golden hash17144800277401907079. - Never rewrite
gauntlet.expectwithout expect-regen discipline (tools/qa-gauntlet/README-expect-regen.md). - Catalog IDs only from tier roster / catalog DB; no CatalogWriteGate mutations without EXTEND-ONLY propose/approve path.
- Never claim
logisticsstress axis proven (GAP-13 config-only).
Read program.md before every forge phase — it is the locked
direction file (Karpathy program.md analog).
Invocation
/qa-gauntlet-forge --run-id <id> --tier N --phase pre|a0|post-oracle|e|final
Prefer /team-qa-gauntlet when coordinating ladder + forge + stress.
Model routing
Use opus only for: ≥5 consecutive stuck discards, CRITICAL corpus conflict,
or multi-tier policy contradiction. Default:
| Phase | Model | Notes |
|---|---|---|
| A0 roster digest | haiku |
|
a0 / A1 candidate draft |
sonnet |
architect Tasks |
| B batch + C oracle CLI | tools only | haiku to summarize exit codes |
post-oracle promote judgment |
sonnet after script scorecard |
Never override hardGatesPass |
e / Phase E |
haiku |
|
final / AAR distill |
haiku → sonnet prose |
opus for stuck / multi-tier conflict |
Layout
production/qa/gauntlet/
corpus/ # committed library
recipes/recipe-catalog.yaml
recipes/recipe-weights.json
stress-axes.yaml
coverage-map.json
hard-cases/
index.yaml
<RUN_ID>/forge/
candidates/ # ephemeral — gitignored
scorecard.json
promote-log.md
mid-tier-plan.yaml
data/scenarios/gauntlet-*.policy.json
Lifecycle phases
pre (before qa-gauntlet A0)
- Hindsight recall bank
qa-gauntlet-forge. - Load corpus coverage-map, recipe-weights, hard-cases, index; optionally rank
stress-*recipes when product wants axis pressure. - Emit
forge/mid-tier-plan.yaml(ranked recipes × coverage gaps, 0–2 hard-cases). - Append plan summary to
forge/promote-log.md.
a0 (with Phase A0/A1)
sim-data-specialist(haiku) roster — prefer under-covered platforms.- Apply top-weighted recipes (
tierMin≤ tier), including stress-axis recipes when selected (stress-weapons-*,stress-ew-*,stress-logistics-config-only). Stress candidates must plan a control sibling for weapons/ew. - Architect Tasks in one turn when disjoint; never invent expect envelopes.
- Validate; invalid → regenerate once, then discard +
FAILED:retain. - T3+ concurrent-thread claims: invoke
/qa-gauntlet-mission-thread(honesty gate; do not duplicate the checklist here).
post-oracle (after Phase C)
- Scorecard first:
python3 tools/qa-gauntlet/forge_scorecard.py --run-dir … --tier N. Never overridehardGatesPass. - If any candidate claimed stress axes: invoke
/qa-gauntlet-stressproof gate (gate_stress_proof.py --axis <id>for each claimed axis — do not default-verify the full catalog on a single-axis candidate); attachstress-proof-report.jsonpath in promote-log. Missing proof for claimed non-config-only axes → do not promote as axis-proven. Logistics remains config-only/unproven in the log. - Promote / discard / hard-case per scorecard; update weights.
e / final
Unchanged adaptive weights, stuck detection, Hindsight retain, corpus commits when improved.
Promotion
Hard gates: catalog resolve; batch stable at tier ticks; determinism; oracle or useful
fail; policy/corpus paths only. On promote: expect-regen, index/coverage, weight up,
commit qa(forge): promote ….
Mutation dimensions
See corpus/recipes/recipe-catalog.yaml (includes stress-* Wave 3 recipes).
Policy JSON only. Stress proof modes: tools/qa-gauntlet/README-stress-axes.md.
Preferred agents
| Agent | Role | Model |
|---|---|---|
qa-lead |
Scorecard interpretation, promote/discard | sonnet |
military-simulation-architect |
Candidate drafting | sonnet |
sim-data-specialist |
Catalog roster | haiku |
hindsight-dev-memory-lead |
Bank qa-gauntlet-forge |
haiku/sonnet |
hindsight-aar-analyst |
Final distillation | haiku/sonnet |
Hindsight bank contract (qa-gauntlet-forge)
Recall every pre; retain on promote/discard/stuck/final. Never from sim Tick().
Wiring from /qa-gauntlet
Required at: pre, a0, post-oracle, e, final.
Success checks
forge/promote-log.mdexists; stress-proof path when axes claimed.- Coverage non-decreasing on promotes; CI gauntlet-oracle smoke + ladder seeds.
- Baltic v2 hash + DelegationBridge zero-touch.
See also
/team-qa-gauntlet— multi-agent entry/qa-gauntlet— ladder owner/qa-gauntlet-ui— game UI Smoke/Pressure (not forge)/qa-gauntlet-combat-ui— engage/kill presentation (not Slice B)/qa-gauntlet-mission-thread— T3+ thread honesty aftera0/qa-gauntlet-agentic-resilience— quarantine / never-LLM-override/qa-gauntlet-stress— axes + proof gate/qa-gauntlet-remediation— Phase D / UCA/team-qa— human sprint package / manual UAT