qa-vision-review
AI vision review of screenshots. Catches visual anomalies DOM probes cannot see — cropped UI, broken icons, misaligned elements, modal positioning errors, color-contrast issues, layout breakage, stuck
SKILL.mdskills/qa-vision-review/SKILL.md
GitHubqa-vision-review
Overview
This is the only skill in Argus that consumes images instead of DOM. Every other detector queries the rendered DOM via browser_evaluate. This skill takes the screenshots produced during Step 5.4 (h) and hands them to a Sonnet sub-agent with vision enabled. The sub-agent looks at what the page actually looks like and flags visual bugs that no DOM probe can see.
The skill is invoked from orchestrator Step 6 after all per-cell execution is complete.
What it checks
Things DOM probes structurally cannot detect, but a human-or-vision-model sees instantly:
| Issue type | Severity | What it catches |
|---|---|---|
croppedElement |
high | Button text, headings, or content visibly cut off (e.g. "Save Cha…" instead of "Save Changes") |
brokenIcon |
high | Icon shows as broken-image placeholder, empty box, or "X" — including missing webfont glyphs |
misalignedElement |
medium | Form labels visibly off-axis from inputs, columns drifting, vertical center broken |
modalPositionWrong |
high | Modal/dialog appears in wrong location — off-screen, partially hidden, or overlapping wrong region |
dropdownClipped |
high | Open dropdown extends past the visible viewport boundary |
zIndexStackingError |
high | Element visibly behind another that should be on top (modal behind backdrop, tooltip behind chrome) |
colorContrastIssue |
medium | Visibly low-contrast text — barely readable against background even at normal viewing |
visualHierarchyBroken |
medium | H1 visually smaller than H2; primary CTA visually less prominent than secondary action |
excessiveWhitespace |
low | Huge empty regions taking >40% of the viewport without purpose |
imageDistortion |
medium | Stretched, pixelated, or improperly scaled image |
loadingSkeletonStuck |
high | Skeleton placeholder or shimmer still visible after the page should have loaded |
blankRegion |
medium | Large content region rendered empty — possible failed fetch or hidden content |
mobileTextOversized |
medium | Text visibly oversized for the mobile viewport — single word taking multiple lines unnecessarily |
overlappingContainers |
high | Two containers visibly overlapping when they should be stacked or adjacent |
cutoffButtonLabel |
high | Button text exceeds its container and is being clipped |
tooltipPositionWrong |
low | Tooltip pointer not pointing at its trigger, or tooltip rendered off-screen |
formFieldCrowded |
medium | Form inputs visually packed too tightly to be usable |
inconsistentSpacing |
low | Visible inconsistency in card grid gaps, list separators, or button rows |
brokenVisualState |
medium | Hover/focus/active state visible when it shouldn't be, or missing when it should |
brandElementBroken |
high | Logo missing, distorted, or rendered as text fallback |
emptyHeaderOrFooter |
low | Header or footer rendered empty when content was expected |
Severity is set by the sub-agent based on user impact, not just visual severity.
Self-skip conditions
Self-skip silently (no error, no finding) if any of these are true:
customize.toml -> vision_review = false- CLI flag
--no-visionwas passed - The cell's
routeis not incustomize.toml -> vision.critical_routes(whencritical_routesis non-empty) - The orchestrator has already reviewed
vision.max_cellscells this run - No screenshots exist for this cell in
.tmp/{runId}/screenshots/{cell.id}-*
Log a single line summary per skipped cell: [skip vision] {cell.id} {reason}.
Orchestrator flow
The orchestrator runs this skill ONCE at Step 6, after all per-cell execution has completed. It does NOT run per-cell during Step 5.
Step 6 vision review (pseudo-code):
1. Read customize.toml -> [vision] section:
enabled = vision_review (top-level, default true)
criticalRoutes = vision.critical_routes (default [])
maxCells = vision.max_cells (default 6)
model = models.vision_model (default "sonnet")
2. If enabled = false OR --no-vision passed:
Log: "⏭️ Vision review disabled — skipping"
Return.
3. Build the cell list to review:
For each cell in audit-plan.json:
a. If criticalRoutes is non-empty AND cell.route not in criticalRoutes → skip
b. Find screenshots: glob .tmp/{runId}/screenshots/{cell.id}-*.png
c. Prefer the *-annotated.png; fall back to the clean *.png
d. If no screenshots → skip
Sort the resulting list by phase (smoke → regression → full) and priority.
Truncate to maxCells.
4. For each cell in the truncated list:
a. Read the chosen screenshot path
b. Read up to 2 representative screenshots per cell (one viewport, max 2)
c. Dispatch a Sonnet sub-agent (see Sub-agent prompt below)
d. Parse the JSON array from the response
e. For each finding, append to .tmp/{runId}/issues/{cell.id}.jsonl
using the schema in qa-argus Step 5.4 (j)
f. Increment visionFindingsCount in run-summary.json
5. If sub-agent dispatch errors (model error, timeout, etc.):
Log warning: "Vision review error on {cell.id}: {message}"
Continue with the next cell. NEVER block the audit on a single vision failure.
6. After the loop, write to run-summary.json:
{
"visionEnabled": true,
"visionCellsReviewed": N,
"visionFindingsTotal": M,
"visionFindingsByType": { croppedElement: 2, brokenIcon: 1, ... }
}
Sub-agent prompt
The orchestrator builds and dispatches this prompt via the Agent tool with vision capability:
Agent({
subagent_type: "general-purpose",
model: "sonnet",
description: "Argus vision review of {cell.id}",
prompt: `
You are reviewing a screenshot of a web page rendered during an automated QA audit.
Cell context:
Route: {cell.route}
Viewport: {cell.viewport.name} ({cell.viewport.width}x{cell.viewport.height})
Browser: {cell.browser}
App URL: {baseUrl}
The screenshot is attached. Look at it carefully and identify ONLY VISUAL anomalies — bugs that are obvious from looking at the rendered page but that an automated DOM query would miss.
What to look for:
croppedElement — button/heading/content visibly cut off
brokenIcon — a broken-image placeholder (failed <img>), a literal "tofu" □ box, or an "X"/empty square where an icon CLEARLY failed to load. This does NOT include caret / chevron / triangle glyphs (^ v < > ‹ › ▲ ▼ ⌃ ⌄ ∧ ∨) used as sort, expand/collapse, dropdown, or pagination indicators — those are valid, intentional UI controls, never a broken icon.
misalignedElement — form labels off-axis from inputs, columns drifting, vertical center broken
modalPositionWrong — modal off-screen, partially hidden, overlapping wrong region
dropdownClipped — open dropdown extending past viewport edge
zIndexStackingError — element visibly behind another that should be on top
colorContrastIssue — barely-readable text against its background
visualHierarchyBroken — H1 smaller than H2; primary CTA less prominent than secondary
excessiveWhitespace — huge empty region taking > 40% of viewport without purpose
imageDistortion — stretched, pixelated, badly scaled image
loadingSkeletonStuck — skeleton or shimmer still visible when page should be loaded
blankRegion — large content area rendered empty
mobileTextOversized — text visibly too large for mobile viewport
overlappingContainers — two containers visibly overlapping when they should be stacked
cutoffButtonLabel — button text exceeds container and is clipped
tooltipPositionWrong — tooltip pointer not pointing at its trigger
formFieldCrowded — inputs packed too tight to be usable
inconsistentSpacing — visible inconsistency in card/list/button gaps
brokenVisualState — hover/focus visible when it shouldn't be, or missing when it should
brandElementBroken — logo missing, distorted, or text-fallback only
emptyHeaderOrFooter — header or footer rendered empty when content expected
Strict rules:
- Only flag issues that are CLEARLY visible. Do NOT speculate.
- DO NOT flag minor design choices, brand-specific styling, or stylistic preferences.
- DO NOT flag things that are arguably intentional (e.g. dark backgrounds, white space as design).
- DO NOT re-detect DOM-level bugs (touch target size, missing alt, missing h1) — those are handled by other detectors.
- Sort indicators from component libraries are STANDARD, intentional UI — never a bug. This includes MUI `TableSortLabel` (a small up/down arrow ▲/▼ SVG), Ant Design, Bootstrap, PrimeNG, AG Grid sort carets/chevrons (^ v ‹ › ⌃ ⌄ ∧ ∨), etc. NEVER flag any of these as broken, missing, wrong, low-quality, or "should be an arrow" — the glyph shape (caret vs arrow) is a stylistic preference, which rule 2 already forbids flagging.
- CRITICAL for sort columns: MUI and most table libraries show the sort indicator ONLY on the currently-active (and sometimes hovered) column — inactive columns intentionally have NO visible arrow. A static screenshot captures no hover state, so most columns will correctly show no indicator. Do NOT flag a column as "missing sort indicator/arrow" — the absence is by design, not a bug. Sort affordance is a DOM/interaction concern handled by qa-test-data-controls, not vision.
- Limit to 10 findings total. Pick the most impactful.
- Output ONLY a valid JSON array. No prose, no markdown, no preamble.
Output schema (one object per finding):
{
"issueType": <one of the categories listed above>,
"severity": "critical" | "high" | "medium" | "low",
"description": "<plain-English description of the visual issue, max 160 chars>",
"regionHint": "<approximate location: top-left | top-center | top-right | middle-left | middle | middle-right | bottom-left | bottom-center | bottom-right>",
"selectorGuess": "<best-guess CSS selector if obvious from text content, otherwise null>"
}
If nothing is visually wrong, return [].
`,
attachments: [<path to chosen screenshot>]
})
The orchestrator passes the screenshot file as an attachment using the Agent tool's image-input capability.
Finding-to-issue mapping
For each finding returned by the sub-agent, build the standard Argus issue record:
{
runId: <currentRunId>,
cellId: <cell.id>,
skill: "qa-vision-review",
issueType: <finding.issueType>,
severity: <finding.severity>,
route: <cell.route>,
viewport: <cell.viewport.name>,
viewportClass: <cell.viewportClass>,
browser: <cell.browser>,
selector: <finding.selectorGuess || null>,
description: <finding.description>,
bbox: null, // vision findings do not have an exact bbox
regionHint: <finding.regionHint>, // extra field
screenshotPath: <path to the reviewed screenshot>,
annotatedScreenshotPath: <same path if it was already annotated, else null>
}
Note: vision findings do NOT receive a new annotation pass — the screenshot the sub-agent reviewed IS the evidence attached to the bug. The regionHint field gives the QA reviewer a quick "where to look."
Append each finding to .tmp/{runId}/issues/{cell.id}.jsonl immediately (streaming write, same as Step 5.4 j).
Cost control
The [vision] block in customize.toml controls cost:
[vision]
critical_routes = ["/", "/login", "/checkout", "/dashboard"]
max_cells = 6
| Config | Behavior |
|---|---|
critical_routes = [] (empty) |
Review every cell up to max_cells |
critical_routes = ["/", "/login"] |
Review only cells whose route exactly matches; ignore others regardless of max_cells |
max_cells = 6 (default) |
Hard ceiling — after 6 cells reviewed, stop emitting vision findings for the rest of the run |
vision_review = false (top level) |
Skill silently no-ops |
Per-cell cost estimate (Sonnet vision):
| Input | Tokens | Cost @ Sonnet 4.6 |
|---|---|---|
| Screenshot (1440×900 PNG, ~1500 tokens after encoding) | 1500 | $0.0045 |
| Prompt + cell context | 700 | $0.0021 |
| Output (JSON findings, ~10 items × 80 tokens) | 800 | $0.012 |
| Total per cell | ~3000 | ~$0.019 |
For the default max_cells = 6: ~$0.11 per audit added. For 12-cell vision on every route: ~$0.23 added.
Hard rules (enforce strictly)
NEVER call
browser_evaluateor any MCP tool. This skill works only from screenshot files on disk. The browser may have been closed by the time Step 6 runs.NEVER re-detect DOM-level bugs. If the vision model wants to flag "touch target too small," reject and skip —
qa-detect-touchalready covers it. Vision is for what DOM probes cannot see.NEVER block the audit on a vision failure. Sub-agent timeout, model error, parse failure — all log and continue.
NEVER ship a finding without a screenshot path. The reviewed screenshot path goes on every emitted finding so the bug-filer has evidence to attach.
NEVER exceed
max_cells. Track the running count in memory across the loop; stop emitting once the cap is hit, even if more cells qualify.NEVER mutate the screenshot file. Read-only. The annotated screenshot was produced in Step 5.4 (h); this skill just consumes it.
Self-skip log messages
For consistent logs the orchestrator should emit one of these per skipped cell:
| Reason | Message |
|---|---|
vision_review = false |
⏭️ Vision review disabled (set vision_review = true to enable) |
--no-vision flag |
⏭️ Vision review skipped (--no-vision flag passed) |
Route not in critical_routes |
⏭️ Vision skipped {cell.id} — route {cell.route} not in vision.critical_routes |
max_cells hit |
⏭️ Vision cap reached (max_cells={N}) — skipping remaining cells |
| No screenshots for cell | ⏭️ Vision skipped {cell.id} — no screenshots found in .tmp/{runId}/screenshots/ |
Output summary
After Step 6 completes, append the following to .tmp/{runId}/run-summary.json:
{
"visionEnabled": true,
"visionCellsReviewed": 6,
"visionFindingsTotal": 14,
"visionFindingsByType": {
"croppedElement": 3,
"brokenIcon": 1,
"modalPositionWrong": 2,
"colorContrastIssue": 4,
"dropdownClipped": 1,
"loadingSkeletonStuck": 1,
"imageDistortion": 2
},
"visionCostEstimateUSD": 0.114
}
The bug filer (Step 7) reads these findings from the cell JSONLs just like any other skill's findings — no special-casing required.
Failure modes and recovery
| Failure | Behavior |
|---|---|
| Sub-agent times out (> 30 s) | Log warning, continue to next cell. Do not retry — vision is not critical. |
| Sub-agent returns non-JSON | Log warning with the first 200 chars of response, continue to next cell. |
Sub-agent returns JSON but with unknown issueType |
Map to closest defined category; if none → emit as visualAnomalyOther with severity low. |
| Screenshot file missing or unreadable | Log warning, skip cell, continue. |
customize.toml -> [vision] section missing |
Use defaults (critical_routes = [], max_cells = 6). |
vision_model not set in [models] |
Default to "sonnet". |
| All cells skipped (empty review list) | Log: ⏭️ Vision review: 0 cells qualified — skipping step. Do not emit a summary. |
Notes
- This skill ships with no external dependencies. The only requirement is that the orchestrator's Agent tool supports image inputs (Sonnet 4.6 does).
- The skill does not modify any screenshots. Annotations (red boxes) are produced by the MCP-driven pipeline (
scripts/annotate-cell-prepare.cjs→browser_navigate+browser_take_screenshot→scripts/annotate-cell-finalize.cjs) in Step 5.4 (h.3), not here. - If you want to add a new visual issue type, add it to both the table above and the sub-agent prompt's "What to look for" list. The orchestrator's mapping is pure passthrough — no code changes needed.
- The
regionHintfield is the closest thing this skill can provide to a bounding box. The bug filer (Step 7) can use it to render a textual "see top-right of the screenshot" hint in the ADO ticket body.