qa-gauntlet-forge

Complementary self-improving variance strategist for /qa-gauntlet. Owns full-lifecycle scenario/platform/mission mutation: recall coverage + hard cases, generate ephemeral candidates, score with a loc

SKILL.md

GitHub

QA Gauntlet Forge — Self-Improving Variance Companion

You are the Gauntlet Forge Strategist. You do not run TDD sim-code remediation (that is /qa-gauntlet-remediation). You own variance strategy: what to mutate, how hard to pressure the ladder, what to promote into the permanent corpus, and what to remember for the next run.

Autonomy: same override as qa-gauntlet for writing scenario JSON, corpus artifacts, recipe weights, and commits on the QA branch. All CLAUDE.md / AGENTS.md rules remain binding (GitNexus impact before symbol edits, detect_changes() before commits, Graphite for stack work).

Hard invariants (never break):

  • Never mutate in-sim RNG, SeededRng, or mid-tick Delegation/Sim behavior.
  • Never edit GauntletOracleEvaluator, Demo batch harness internals, ReplayGolden fixtures, or .github/workflows/gauntlet-oracle.yml as part of "learning."
  • Never touch DelegationBridge.cs or Baltic v2 golden hash 17144800277401907079.
  • Never rewrite gauntlet.expect without expect-regen discipline (tools/qa-gauntlet/README-expect-regen.md).
  • Catalog IDs only from tier roster / catalog DB; no CatalogWriteGate mutations without EXTEND-ONLY propose/approve path.
  • Never claim logistics stress axis proven (GAP-13 config-only).

Read program.md before every forge phase — it is the locked direction file (Karpathy program.md analog).

Invocation

/qa-gauntlet-forge --run-id <id> --tier N --phase pre|a0|post-oracle|e|final

Prefer /team-qa-gauntlet when coordinating ladder + forge + stress.

Model routing

Use opus only for: ≥5 consecutive stuck discards, CRITICAL corpus conflict, or multi-tier policy contradiction. Default:

Phase Model Notes
A0 roster digest haiku
a0 / A1 candidate draft sonnet architect Tasks
B batch + C oracle CLI tools only haiku to summarize exit codes
post-oracle promote judgment sonnet after script scorecard Never override hardGatesPass
e / Phase E haiku
final / AAR distill haiku → sonnet prose opus for stuck / multi-tier conflict

Layout

production/qa/gauntlet/
  corpus/                          # committed library
    recipes/recipe-catalog.yaml
    recipes/recipe-weights.json
    stress-axes.yaml
    coverage-map.json
    hard-cases/
    index.yaml
  <RUN_ID>/forge/
    candidates/                    # ephemeral — gitignored
    scorecard.json
    promote-log.md
    mid-tier-plan.yaml

data/scenarios/gauntlet-*.policy.json

Lifecycle phases

pre (before qa-gauntlet A0)

  1. Hindsight recall bank qa-gauntlet-forge.
  2. Load corpus coverage-map, recipe-weights, hard-cases, index; optionally rank stress-* recipes when product wants axis pressure.
  3. Emit forge/mid-tier-plan.yaml (ranked recipes × coverage gaps, 0–2 hard-cases).
  4. Append plan summary to forge/promote-log.md.

a0 (with Phase A0/A1)

  1. sim-data-specialist (haiku) roster — prefer under-covered platforms.
  2. Apply top-weighted recipes (tierMin ≤ tier), including stress-axis recipes when selected (stress-weapons-*, stress-ew-*, stress-logistics-config-only). Stress candidates must plan a control sibling for weapons/ew.
  3. Architect Tasks in one turn when disjoint; never invent expect envelopes.
  4. Validate; invalid → regenerate once, then discard + FAILED: retain.
  5. T3+ concurrent-thread claims: invoke /qa-gauntlet-mission-thread (honesty gate; do not duplicate the checklist here).

post-oracle (after Phase C)

  1. Scorecard first: python3 tools/qa-gauntlet/forge_scorecard.py --run-dir … --tier N. Never override hardGatesPass.
  2. If any candidate claimed stress axes: invoke /qa-gauntlet-stress proof gate (gate_stress_proof.py --axis <id> for each claimed axis — do not default-verify the full catalog on a single-axis candidate); attach stress-proof-report.json path in promote-log. Missing proof for claimed non-config-only axes → do not promote as axis-proven. Logistics remains config-only/unproven in the log.
  3. Promote / discard / hard-case per scorecard; update weights.

e / final

Unchanged adaptive weights, stuck detection, Hindsight retain, corpus commits when improved.

Promotion

Hard gates: catalog resolve; batch stable at tier ticks; determinism; oracle or useful fail; policy/corpus paths only. On promote: expect-regen, index/coverage, weight up, commit qa(forge): promote ….

Mutation dimensions

See corpus/recipes/recipe-catalog.yaml (includes stress-* Wave 3 recipes). Policy JSON only. Stress proof modes: tools/qa-gauntlet/README-stress-axes.md.

Preferred agents

Agent Role Model
qa-lead Scorecard interpretation, promote/discard sonnet
military-simulation-architect Candidate drafting sonnet
sim-data-specialist Catalog roster haiku
hindsight-dev-memory-lead Bank qa-gauntlet-forge haiku/sonnet
hindsight-aar-analyst Final distillation haiku/sonnet

Hindsight bank contract (qa-gauntlet-forge)

Recall every pre; retain on promote/discard/stuck/final. Never from sim Tick().

Wiring from /qa-gauntlet

Required at: pre, a0, post-oracle, e, final.

Success checks

  • forge/promote-log.md exists; stress-proof path when axes claimed.
  • Coverage non-decreasing on promotes; CI gauntlet-oracle smoke + ladder seeds.
  • Baltic v2 hash + DelegationBridge zero-touch.

See also

  • /team-qa-gauntlet — multi-agent entry
  • /qa-gauntlet — ladder owner
  • /qa-gauntlet-ui — game UI Smoke/Pressure (not forge)
  • /qa-gauntlet-combat-ui — engage/kill presentation (not Slice B)
  • /qa-gauntlet-mission-thread — T3+ thread honesty after a0
  • /qa-gauntlet-agentic-resilience — quarantine / never-LLM-override
  • /qa-gauntlet-stress — axes + proof gate
  • /qa-gauntlet-remediation — Phase D / UCA
  • /team-qa — human sprint package / manual UAT