embedding-quality-scanner
Embedding quality scanner: audit chunking strategy, model selection, and embedding configuration. Read-only. Trigger: /embed-quality
SKILL.mdskills/ai-ml/embedding-quality-scanner/SKILL.md
GitHub/embed-quality
RAG and semantic search depend on embedding quality. This skill finds configuration issues before they produce bad retrieval results.
What this is for
- Documents embedded without chunking strategy (context dilution)
- Query and document embeddings from different models (semantic mismatch)
- Chunk size inappropriate for content type (too large = noise, too small = lost context)
- Read-only skill. No code changes.
PROTECTION RULE - never ~/.claude/
Read-only skill. Guard required if write mode added later.
What You Must Do When Invoked
During analysis, assign a confidence level to each finding: proven (confirmed by evidence), likely (strong signal, needs review), or suspected (weak signal).
If /embed-quality -help or /embed-quality -h (without further arguments)
is invoked: output the ## Usage section unchanged and stop.
Otherwise follow these steps in order, skipping none.
Step 1 - Help check
If invoked with -help or -h, output the ## Usage section unchanged and stop.
Step 2 - Clarify target
Clarify -ProjectDir. Get confirmation.
Step 3 - Scan
& "<SKILL_DIR>/scripts/embed-scan.ps1" -ProjectDir "<path>"
Step 4 - Classification
Read each embedding config:
- Consistent: appropriate chunk size, model parity, adequate overlap
- Suboptimal: room for improvement but functional
- Mismatched: query/doc model difference, no overlap
- Broken: will produce bad retrieval results
Step 5 - Write report
File embedding-quality-report.md in current working directory:
- Summary - embeddings by classification.
- Embedding table - broken first. Per embedding: file, line, model, chunk size, overlap, has model parity, recommendation.
- Open questions.
Step 6 - Summarize
State report path, highlight embeddings that will produce bad retrieval.
Usage
/embed-quality # interactive
/embed-quality <dir> # scan project
/embed-quality -help