claim-evidence-gate
Use when validating claims against primary sources, empirical data, or citation evidence prior to submission or peer review
SKILL.mdskills/en/claim-evidence-gate/SKILL.md
GitHubClaim & Evidence Gate Skill
Purpose
Apply invariant validation discipline to verify that every thesis claim in the manuscript is directly backed by primary sources, empirical data, or validated citations. This gate prevents unevidenced assertions, quote mining, and modality-evidence misalignment from reaching submission.
Trigger Conditions
- After completing a draft of each chapter or major section
- During the final pre-submission review phase
- When a reviewer or collaborator raises concerns about evidentiary support
For diachronic claims (transmission, juxtaposition, continuity, novelty), check the diachronic-claim-typing inventory. Do not PASS transmission (T3) on similarity alone.
Evaluation Steps
Step 0: Literature Grounding Prerequisite (added in v1.3.2)
- If
python3 scripts/check_literature_grounding.pyreturns FAIL, halt this gate and return tocitation-traceability-auditStep 2.5 (literature ingestion). - WARN-only results may proceed (full-text grounding recommended before submission).
Step 1: Extract Claim–Evidence Pairs
- Identify paragraph blocks containing claims, statistical figures, or calculated values (ratios, percentage changes, growth rates).
- Note: Execute
python3 scripts/check_fact_grounding.pyas a pre-filter candidate extractor.
- Note: Execute
- Pair each claim with its corresponding primary data, citation, footnote, or
docs/data/table ID.
Step 2: Seven-Axis Rubric & Scoring
Score each claim–evidence pair into PASS, WARN, FAIL, or N/A across seven axes:
| Evaluation Axis | PASS (Valid) | WARN (Tone Adjustment Needed) | FAIL (Unsubstantiated / Rejected) |
|---|---|---|---|
| Direct Match | Cited data directly substantiates claim, and calculated figures have source markers and raw values recorded in text, footnotes, or docs/data/ |
Evidence is indirect, or raw figure reference is partially ambiguous | Cited evidence contradicts claim, calculated figure lacks source marker, or figure source is unidentifiable |
| Terminological Precision | Key terms and numbers are accurately represented | Terminology interpretation is slightly stretched | Data/text is distorted or over-interpreted |
| Context Safety | Source context is fully preserved | Contextual framing is partially insufficient | Claim relies on decontextualized quote mining |
| Modality Alignment | Claim tone matches evidence strength | Modality is slightly too strong for sample size | Overgeneralized despite weak sample without hedging |
| Benchmark Grounding | Comparative or degree evaluation terms include objective benchmarks at first occurrence | Comparative benchmark is partially insufficient or reference is ambiguous | Evaluative terms used (e.g. "high", "pronounced") without any benchmark |
| Internal Heterogeneity | Internal differentiation of the analyzed group (skill level, region, gender, etc.) is appropriately addressed | Group is treated as homogeneous but heterogeneity is acknowledged in footnotes | Macro aggregates only; internal group differentiation is ignored |
| Field Disagreement | Contested claims acknowledge opposing views in the text, or the proposition is confirmed as consensus / replicated in literature-matrix.md |
The point is contested (contested / contradicted) but opposing views appear only in footnotes, or tone remains slightly too strong despite acknowledgment |
A contested proposition is asserted on a single paper with no mention of opposing views, or a contradicted / retraction-watch source is cited as established knowledge |
⚠️ N/A (Not Applicable) Rules
- Internal Heterogeneity Axis: For claims that do not target social groups or specific categories (e.g., conceptual analysis, textual/philological interpretation, chronological dating, mathematical proofs), score this axis as N/A.
- Benchmark Grounding Axis: For purely factual descriptive statements that contain no degree or comparative evaluative terms (e.g., "The treaty was signed in 1945"), score this axis as N/A.
- Field Disagreement Axis: For factual statements where dispute cannot arise (dates, reported values from official statistics, etc.) or for claims proposing genuinely novel concepts with no prior literature, score this axis as N/A. Also score N/A when the contestation status is
unknown; however, if a load-bearing claim (research question, core causal claim) remainsunknown, recommend runningliterature-searchStep 1.5 (Faction Discovery).
Benchmark Grounding Axis Detailed Rules
When using comparative or degree terms such as "high", "low", "significant", "pronounced", "substantial", "large", "majority", or "pronounced increase" at first occurrence, at least one of the following benchmarks must be explicitly specified:
- Social Sciences / Development Economics: International institutional thresholds (e.g., World Bank / UNDP warning line of 40.0 Gini), regional/national averages (e.g., ASEAN mean).
- Natural Sciences / Empirical Data: Statistical significance thresholds (e.g., $p < 0.05$), effect sizes (e.g., Cohen's $d > 0.8$), control group baseline values.
- Humanities / History: Standard critical edition text baseline, recognized periodization/baseline indicators.
- Reference Formats: Section anchor (
#1.2.3), section cross-reference (cf. Section X.X.X), or footnote ([^1]). Subsequent occurrences may refer back to the initial benchmark.
Step 3: Quality Gate Thresholds & Low Confidence Criteria
- Active Axes Definition: All axes excluding those marked N/A.
- Low Confidence Criterion: Flagged if 1 or more active axes receive WARN or FAIL.
- Overall Gate Pass/Fail Rules:
- 🟢 PASS: All active axes PASS. No changes required.
- 🟡 WARN (Soften Tone / Add Benchmark): 0 FAILs, but 1+ WARNs. Specify the exact reason (e.g., Tone Mismatch vs Missing Benchmark) and recommend revisions.
- 🔴 FAIL (Requires Evidence / Revision): 1+ FAILs. Identify the specific failure mode (Direct Match failure, ungrounded figure, Tone vs Benchmark) and require supplementary data, citation replacement, or claim removal.
Step 4: Generate Remediation Report
Produce docs/design/evidence-gate-report.md detailing scores and actionable recommendations.
⚠️ Output Boundary Rule
In manuscript text (docs/chapters/), inline citations MUST use standard academic styles (e.g., (PSA, 2024) or (World Bank, 2026), footnotes [^1]).
- Prohibition: Printing local repository file paths (such as
data: docs/data/filename.md#tbl-1) inside citation parentheses in manuscript text (docs/chapters/) is strictly forbidden. - Data Path Storage: Local file paths and table IDs must be recorded and managed exclusively in
docs/data/*.mdtable notes or insidedocs/design/evidence-gate-report.md.
Outputs
docs/design/evidence-gate-report.md(Evidence Integrity Audit Report)