opus-deep-audit
Deep-audit chains in dom_reachable.jsonl (fallback hot.jsonl) via Sonnet→Opus advisor pipeline. Writes opus/<chain_id>.md with verdict + PoC. Respects opus_budget from memory.md. Run after dom-xss-hun
SKILL.md.claude/skills/opus-deep-audit/SKILL.md
GitHubopus-deep-audit
Purpose
Use Opus (via TLX's bounded advisor) to apply real reasoning to chains that survived static + reachability filtering. This is where TP rate gets earned.
Inputs
targets/<name>/chains/dom_reachable.jsonl(preferred) orchains/hot.jsonl(fallback if dom hunt was skipped).- budget:
memory.md > opus_budget_per_target_usd.
Steps
- Read budget. If
status.json.opus_cost_usdalready at limit, halt and require user to raise.
1a. Long-context viability probe (T1.3). Run:
python3 bin/source_token_count.py targets/<name>
The script estimates token count of targets/<name>/sources/
(heuristic: bytes / 4) and writes
status.json.long_context.eligible. When eligible (≤ 800k tokens
by default), it also writes
targets/<name>/opus/whole_tree_prompt.md — a manifest +
instructions you can hand to a Claude Code subagent
(/feature-dev, dispatch on Opus long-context) to perform a
bundle-wide pass without docs_query retrieval.
When `eligible == true`:
- Skip `docs_query` retrieval in Step 2c. The whole tree fits in
conversation context.
- For the gap-audit pass (`opus-gap-audit`), prefer dispatching the
whole-tree prompt over the chain-by-chain audit — Opus reads the
entire bundle and surfaces gaps holistically.
Per the API-key whitelist, this branch runs as Claude Code in your
interactive session. Do NOT add a new `anthropic.Anthropic` code
path to drive long-context audit autonomously.
1b. Two-tier cascade gate (T1.2). Before running any per-chain
audit, gate the input list through bin/run_cascade.py:
python3 bin/run_cascade.py targets/<name>
The cascade combines a deterministic dead-code check with a Sonnet
triage prompt (rubric: literal-constant source, wrapped safe
sanitizer, pure-intermediates-only, short-length gate — see
wiki/tools/karpathy/js-review-cascade.md). It writes:
- chains/survivors.jsonl — chains the gate flagged escalate
- chains/rejected.jsonl — chains the gate flagged reject
- chains/cascade_stats.json — counts + cost
- status.json.phases.cascade and status.json.cascade_cost_usd
Use `chains/survivors.jsonl` as the input to Step 2.
For each chain in `chains/rejected.jsonl`, write a stub
`targets/<name>/opus/<chain_id>.md` containing the gate verdict
and `verdict: false_positive`, reason `gate: <dead_code|sonnet_reject>`.
Then submit via `js_submit_finding(chain_id, verdict='false_positive',
proof='gate: ...')` so the verdict store reflects it.
Do NOT call `js_run_audit` for rejected chains. (2026-05 — Opus
budget rule; reaffirmed after the cascade-gate landed.)
- For each chain in
chains/survivors.jsonl: a.js_examine_chain(chain_id=...)→ full source/sink/path. b.js_get_snippet(qname=...)for each function in the path adjacent ±1 hop. c. Compose the audit context (chain detail + snippets + framework hints + relevantwiki/techniques/<sink_kind>/page if exists — retrieve viadocs_queryagainst the wiki collection). d. Triggerjs_run_audit(target_folder=...)if not already running for this target (it's a fire-and-forget audit loop that uses the Sonnet→Opus advisor internally). e. Polljs_audit_status(run_id=...)untilstatus == "done". f. Pull verdict + reasoning + PoC from audit output. - Write
targets/<name>/opus/<chain_id>.mdwith sections:- Chain summary
- Source / sink / path
- Audit verdict
- Reasoning trace (summarized — full transcript is in TLX SQLite)
- Proposed PoC (HTML / curl / browser steps)
- Submit verdict via
js_submit_finding(chain_id, verdict, proof). - Track cost: read
js_audit_statuscost fields, add tostatus.json.opus_cost_usd. The cascade gate's Sonnet cost is tracked separately atstatus.json.cascade_cost_usdso the Opus budget cap remains an Opus budget.
Outputs
targets/<name>/chains/{survivors,rejected}.jsonl(Step 1b).targets/<name>/chains/cascade_stats.json(Step 1b).targets/<name>/opus/<chain_id>.mdper chain (rejected stubs + audited).- updated
status.json.phases.{cascade,opus}.
Failure modes
- Budget exceeded mid-batch → halt, write partial result block, notify user.
js_run_auditerrors out → log tostatus.json.errors, mark chainverdict: undeterminedsoautoresearch-loopcan retry later.- Opus verdict says TP but proposed PoC obviously wrong → write
verdict but flag
requires_user_review: truein result block. - Cascade gate parse failure (
parse_failures > 0in stats) → those chains default toescalate(bias toward Opus). Inspect the gate payload'serrorfield on each survivor's_gateannotation. - Cascade gate rejects a chain that the user later confirms TP →
feedback loop: add the chain pattern to the cascade rubric in
wiki/tools/karpathy/js-review-cascade.mdand re-run.
Result block
"phases": {
"cascade": { "status": "done", "ts": "<iso>",
"total": N, "rejected": R, "escalated": E,
"rejected_dead_code": D, "rejected_sonnet": S,
"gate_cost_usd": Decimal, "gate_duration_ms": N,
"parse_failures": F, "input": "chains/...jsonl" },
"opus": { "status": "done", "ts": "<iso>",
"chains_audited": N, "tp": X, "fp": Y, "undetermined": Z,
"opus_cost_usd": Decimal, "user_review_flagged": [...] }
}