auto-skill-build-mcp-conformance-harness

Design a repeatable MCP conformance, health-check, and tool-surface regression harness for local or CI use. Use when the user asks for MCP CI, protocol or OAuth conformance, server doctor checks, tool

SKILL.md

GitHub

MCP Conformance Harness

Generated by: Codex Supercharge maintenance automation.

Goal: turn MCP server testing into a repeatable harness with clear evidence, secret handling, and pass/fail gates.

Workflow

  1. Use $sandbox-source-intake and $mcp-safety-review before running a fresh MCP server, unknown CLI, install script, or broad-account tool.
  2. Identify the target matrix:
    • transport: HTTP or stdio
    • auth: none, static token, OAuth interactive, OAuth headless, or M2M
    • checks: doctor, tool surface, protocol, OAuth, apps UI
  3. Prefer read-only checks first: server probe, server doctor, and server export with --no-telemetry and JSON output.
  4. Add protocol, oauth, or apps conformance suites only after auth and blast radius are understood; emit JUnit/XML or JSON summary artifacts.
  5. Capture before/after server export snapshots and sort them before diffing when release tool-surface drift is the risk.
  6. Use the command recipes in references/mcp-conformance-harness.md instead of expanding long shell blocks into the skill body.
  7. Store reports as CI artifacts, but keep credentials out of artifacts. Prefer environment secrets, credentials files with mode 0600, or CI secret stores.
  8. Summarize failures by phase: transport, auth, protocol, tool schema, apps UI, or release diff.

Harness Contract

# MCP Harness Plan
Target:
Transport/auth:
Read-only checks:
Conformance suites:
Tool-surface diff:
Secrets:
Artifacts:
Pass/fail gates:
Known skips:

Skip When

  • The user only wants to manually explore one server once.
  • No MCP endpoint, stdio command, or runnable server target is available.
  • The server/tool is untrusted and has not passed source intake or safety review.
  • Running the harness would call write/destructive tools without explicit scope.

References

  • references/mcp-conformance-harness.md
  • Source inspiration only, code not executed: sources/mcpjam-inspector
  • Support skills: $mcp-server-lab, $mcp-safety-review, $auto-skill-safety-mcp-secret-guard

Validation

plugins/codex-supercharge/scripts/skill_audit.sh plugins/codex-supercharge/skills
python3 "$HOME/.codex/skills/.system/skill-creator/scripts/quick_validate.py" \
  plugins/codex-supercharge/skills/auto-skill-build-mcp-conformance-harness