engagement-audit
On-site engagement audit for visitors arriving from AI answers. Use when AI referral traffic bounces or fails to convert. Checks answer-first openings, mid-funnel context, interstitials, scannability,
SKILL.mdskills/engagement-audit/SKILL.md
GitHubEngagement (on-site layer)
The second half of the problem, and a genuinely different one. Everything upstream asks whether a machine can find, read and use the page. This asks whether the human the machine sent stays.
The arrival pattern that makes this different
A visitor from an AI answer does not arrive like a search visitor. They arrive:
- Mid-funnel — on a deep page, not the homepage, because the assistant cited the page that answered the question.
- Partially informed — they already have an answer. They are here to verify it, or to act on it.
- Without the navigation context a site assumes people accumulate on their way in.
So a deep page that reads as a fragment of a journey — assuming you know what
the product is, who the company is, and what the previous page said — fails
this visitor even when the site works fine for someone who entered at the
homepage. EN002 is the check for exactly that.
When to use
After crawl-reachability. Run standalone when a site gets AI-referred traffic
that does not convert or stay.
What this layer can and cannot see
It reads shipped markup only, so it measures page-side proxies for engagement, never engagement itself. Two consequences are load-bearing:
- No JavaScript is executed and no session is simulated. Anything that only exists after hydration is out of scope, and a finding is never raised about it.
- Where a proxy stands in for an unmeasurable quantity, the finding says so.
EN009is labelled a structural proxy rather than presented as a Core Web Vitals measurement.
Inputs
- Required — a run workspace from the two upstream stages:
preflight.jsonfor the scope domain andreachability.jsonfor the page inventory and saved HTML paths. - Optional —
--config(anAuditConfigJSON) and--strictness(lenient|balanced|strict) to override interstitials and chrome-ratio thresholds. Defaults come fromAuditConfig. - Not consumed — no network access, no analytics, no prior stage's findings, no visitor data. The layer is re-runnable offline and deterministic.
python3 scripts/check_engagement.py [--workspace PATH] [--json]
Procedure
Deterministic, in this order. Every check is a pure function of the fetched markup plus the config; nothing is sampled and nothing depends on the clock.
- Load the inventory and pages. Resolve the scope domain from
preflight, then read every fetched page; missing HTML becomes a coverage limitation. - Answer-first orientation (
EN001). Test whether the opening states the answer or sells at the visitor. - Intent continuity (
EN002). Test whether a deep page stands alone for a mid-funnel arrival who never saw the homepage. - Interstitials (
EN003). Detect anything demanding an interaction before the first one the visitor came for. - Scannability (
EN004). Checkh1presence and heading-level continuity. - Chrome ratio (
EN005). Compare substantive content against navigation, footer and consent chrome. - Substance in images (
EN006). Flag content carried only by images with no text alternative. - Form friction (
EN007). Measure how much is asked before anything is given on conversion paths. - Dead ends (
EN008). Flag pages offering no next step beyond site-wide navigation. - Perceived speed (
EN009). Count render-blocking resources in<head>, and label the result a structural proxy. - Attach the sample caveat. Every population claim carries its sample size for the advisory layer to verify.
Checks and severities
Every check the procedure above can emit, with the claim level it is allowed to assert.
| Check | Detects | Severity | Claim level |
|---|---|---|---|
EN001 |
The opening sells rather than answering | medium | 3 |
EN002 |
Deep pages that do not stand alone for a mid-funnel arrival | medium | 3 |
EN003 |
Interstitials blocking the first interaction | medium | 3 |
EN004 |
Missing h1 or skipped heading levels |
medium | 1 |
EN005 |
Substance outweighed by surrounding chrome | medium | 3 |
EN006 |
Substance carried only in images with no text alternative | medium | 3 |
EN007 |
Conversion forms asking a lot before giving anything | medium | 1 |
EN008 |
Pages offering no next step beyond site-wide navigation | medium | 2 |
EN009 |
Render-blocking resources in <head> (structural proxy) |
medium | 3 |
Honesty about what cannot be measured
EN009 is deliberately labelled a structural proxy, not a measurement.
Core Web Vitals cannot be computed from raw HTML: Largest Contentful Paint,
Interaction to Next Paint and Cumulative Layout Shift all require a real
rendering engine and, for field data, real users.
What is observable is render-blocking resource count in <head> and images
lacking intrinsic dimensions — structural conditions associated with poor LCP
and CLS. The check reports those as what they are, at claim level 3, capped at
medium. Reporting a fabricated performance score from raw HTML would be
precisely the kind of confident-and-wrong finding this marketplace is built to
avoid.
EN008 needs at least three pages to distinguish site-wide chrome links from
page-specific next steps, and skips itself with a stated note below that
threshold rather than guessing.
Summariser legibility
Handout appendix F describes a related failure: when the real substance of a
message is not available as readable text — carried in an image, or surrounded
by low-value filler — a summariser has little to work with and the important
part disappears. EN005 and EN006 cover both shapes of that on a web page.
Output
06-engagement-audit.json with per-page opening analysis, interstitial
inventory, heading hierarchy, chrome ratio, form field counts, next-step link
analysis and the findings.
Guardrails
Read-only; consumes saved HTML only. No form is ever submitted, no interaction is simulated, no JavaScript is executed.
Grounding
Handout appendix F (summariser legibility) and appendix E (answers are shaped by who is asking, so a page must stand alone for an arrival it did not script). The mid-funnel arrival pattern follows from the citation model in arXiv:2604.25707: the cited page is the page that answered the question, which is rarely the homepage.