hermes-compatibility-audit
Read-only compatibility audit between `hermes-agent-setup` and the latest stable upstream `hermes-agent` release plus official Hermes docs. Use whenever the task is to decide whether this repository s
SKILL.md.claude/skills/hermes-compatibility-audit/SKILL.md
GitHubHermes Compatibility Audit
Purpose
Use this skill to answer one practical question:
is hermes-agent-setup still compatible with the current stable upstream
hermes-agent contract or not?
This is a read-only compatibility gate. The job is to compare official upstream Hermes behavior against the assumptions encoded in this repository and return a clear verdict, not to design or apply a migration.
Treat the repository's current runtime baseline as a repository fact, not as
current upstream truth. Today the shipped helper targets latest-only
hermes-agent v2026.5.16 / v0.14.0 or newer; the historical PRD baseline
was v2026.4.13 / v0.9.0, but that legacy baseline is no longer the
supported public runtime contract.
Scope
Cover the repository's current and planned Hermes-facing contract, especially:
- config location, resolution, and precedence assumptions for
~/.hermes/config.yaml,~/.hermes/.env,HERMES_HOME, profiles, and any official config-path or env-path surfaces - custom-endpoint wiring through
providers.gonkagate,model.provider = gonkagate,model.default,key_env,transport,discover_models, and old direct custommodel.*cleanup - model and provider selection assumptions around
hermes model,provider:modelsyntax, curated model choice, and custom provider behavior - auth and secret-handling assumptions around
OPENAI_API_KEY, unsupported legacy endpoint env such asOPENAI_BASE_URL,auth.json, credential pools, and the repository's decision to keep secrets in~/.hermes/.envrather thanconfig.yaml - workflow and CLI assumptions documented by this repository, such as
hermes model,hermes config set,hermes config path,hermes config env-path,hermes setup,hermes doctor, and profile selection behavior - read-side conflict surfaces described by the PRD, including matching custom
credential pools,
cron/jobs.json, inherited process env, and managed-install or blocked-write behavior - newly required settings, renamed fields, removed commands, or release-level behavior changes that would make the documented GonkaGate Hermes plan stale or unsafe
Default compatibility target:
- latest stable upstream
hermes-agentrelease tag
Secondary watch target:
- newer prerelease or unreleased main-branch signals, but only as an early-warning watchlist unless the user explicitly asks for prerelease compatibility
Boundaries
Do not:
- modify repository code or docs
- broaden product scope beyond the current GonkaGate Hermes contract
- treat the PRD's current release baseline as proof that latest upstream still matches it
- propose shell profile mutation, arbitrary custom base URLs, secret storage in
config.yaml, or direct mutation ofauth.jsonas the default integration path unless the user explicitly asks for a product change - use secondary summaries when primary sources are available
- treat prerelease drift as a stable compatibility failure unless the user explicitly asked to audit prereleases
- turn the audit into an auto-remediation or full migration plan
Primary-Source Discipline
Use primary sources only:
- official Hermes docs, especially:
https://hermes-agent.nousresearch.com/docs/getting-started/installation/https://hermes-agent.nousresearch.com/docs/user-guide/configuration/https://hermes-agent.nousresearch.com/docs/integrations/providers/https://hermes-agent.nousresearch.com/docs/reference/cli-commands/https://hermes-agent.nousresearch.com/docs/reference/profile-commands/https://hermes-agent.nousresearch.com/docs/user-guide/features/credential-pools/
- official upstream release source for the exact stable tag, such as:
https://github.com/NousResearch/hermes-agent/releases- the matching tagged source under
https://github.com/NousResearch/hermes-agent
- shipped stable source, tests, or CLI help for the same stable tag when docs are incomplete or ambiguous
Prefer this discovery order:
- official latest stable release tag from GitHub releases
- official Hermes docs for configuration, providers, CLI, profiles, and credential handling
- tagged upstream source for the exact stable version
- tagged source tests or code paths when docs are incomplete
- isolated CLI help or read-only inspection when docs and source are still insufficient
Useful starting points:
https://github.com/NousResearch/hermes-agent/releaseshttps://hermes-agent.nousresearch.com/docs/user-guide/configuration/https://hermes-agent.nousresearch.com/docs/integrations/providers/https://hermes-agent.nousresearch.com/docs/reference/cli-commands/https://hermes-agent.nousresearch.com/docs/reference/profile-commands/
If official docs and the tagged stable source disagree, trust the tagged stable source or shipped stable behavior and call out documentation drift explicitly.
Safe Read-Only Execution
Keep the audit read-only.
- Prefer docs, release notes, tagged source, source tests, and CLI help over running stateful commands.
- Never run upstream Hermes commands against the user's real
~/.hermes. - If you need CLI help or read-only behavior inspection, isolate it in a
disposable temp directory and point
HOME,HERMES_HOME,XDG_CONFIG_HOME,XDG_DATA_HOME, and any other relevant config roots at temp paths. - Do not run setup flows, login-like flows, or anything that mutates real state.
- Treat isolated local execution as a last resort after docs, release source, and tagged code inspection.
Repository Surfaces To Compare
Start from the current repository contract surfaces:
README.mdAGENTS.mddocs/how-it-works.mddocs/security.mddocs/specs/hermes-agent-setup-prd/spec.mdsrc/cli.tssrc/constants/contract.tspackage.jsonrelease-please-config.jsontest/cli.test.tstest/package-contract.test.tstest/docs-contract.test.tstest/skills-contract.test.ts
Inspect local skills when they encode product assumptions that affect the audit, especially:
.agents/skills/coding-prompt-normalizer/- this compatibility-audit skill itself, if its assumptions look stale
If the repository later adds implementation modules, inspect those too instead of stopping at docs. In particular, compare any future surfaces under:
src/install/src/constants/- config-reading or config-writing modules
- secret-handling helpers
- model qualification artifacts
- runtime verification flows
Upstream Evidence To Gather
For the target stable release, gather evidence for:
- the exact stable tag or version, matching release source, and publish date
- whether official docs and the stable release tag agree on configuration, profiles, provider wiring, and command surfaces
- where Hermes loads global config from and how
HERMES_HOME, profiles, and any official config-path commands affect target resolution - the official contract for named custom providers,
gonkagate,model.default,key_env,transport,discover_models, and direct custom cleanup fields - whether current Hermes guidance still routes secrets to
.envand non-secret config toconfig.yaml - whether unsupported legacy endpoint env such as
OPENAI_BASE_URL,OPENAI_API_KEY,auth.json, credential pools, orcron/jobs.jsonremain active upstream surfaces that could affect the latest-only helper contract - whether managed installs or blocked-write modes remain relevant to a local mutation helper
- whether Hermes added or removed CLI surfaces relevant to this repository's documented flow
- whether release notes or stable source mention changes to config precedence, provider wiring, profile semantics, auth handling, or command surfaces
- any newly required settings, schema migrations, or structural requirements that this repository does not currently satisfy
When searching source or docs, start with these literals:
~/.hermes/config.yaml~/.hermes/.envHERMES_HOMEprovider: customgonkagatecustom_providersOPENAI_API_KEYOPENAI_BASE_URLauth.jsoncron/jobs.jsoncredential_poolhermes modelhermes config sethermes config pathhermes config env-pathhermes setuphermes doctorprofilebase_urltransportdiscover_models
Workflow
- Identify the audit target.
- Determine the latest stable upstream
hermes-agentrelease tag from the official releases page. - Confirm the matching repository source for that stable tag.
- Note any prerelease or main-branch signals separately, but keep them out of the stable verdict unless the user asked for them.
- Determine the latest stable upstream
- Capture the upstream contract before judging compatibility.
- Read official configuration, providers, CLI, profile, and credential docs.
- Read tagged source when docs are vague, incomplete, or omit exact field or command behavior.
- Use isolated CLI help only when docs and source still leave an important ambiguity.
- Map the repository's assumptions.
- Read
README.md,AGENTS.md, anddocs/first. - Then inspect
src/cli.ts,src/constants/contract.ts, release config, tests, and any implementation surfaces that exist. - Keep current shipped-runtime truthfulness separate from broader future product changes.
- Read
- Compare the critical seams one by one.
Config locations and precedenceCompare upstream config and env behavior against the repo's~/.hermes/config.yaml,~/.hermes/.env, and secret-boundary assumptions.Profile and home resolutionCompare upstreamHERMES_HOME, profile, and path-resolution behavior against the repo's target-home assumptions.Provider wiringCompare upstream custom-provider expectations against the repo's plannedgonkagate, canonical base URL, and named-provider usage.Auth and conflict surfacesCompare upstream auth and env behavior against the repo's planned use of.env,OPENAI_API_KEY,OPENAI_BASE_URL,auth.json, credential pools, andcron/jobs.jsondetection.Workflow and command surfacesCompare upstream CLI surfaces and workflows against what this repo promises users today.Managed installs and write-path safetyCompare upstream blocked-write or managed-mode behavior against the repo's planned fail-fast assumptions.Recent release driftCompare the latest stable release notes and stable source against the repo's setup plan.
- Classify the evidence.
- Label each material point as:
confirmed upstream change,confirmed still compatible,confirmed repo-overstatement, orinferred risk. - Keep observed upstream facts separate from your interpretation of impact.
- Label each material point as:
- Decide the verdict.
compatibleNo confirmed upstream stable change breaks the repository's current or planned Hermes contract.compatible with caveatsNo confirmed stable break yet, but there is meaningful ambiguity, documentation drift, or repository overstatement that weakens confidence.incompatibleA confirmed upstream stable change conflicts with a required repository assumption or makes the documented GonkaGate Hermes plan stale or unsafe.
- Name the minimum follow-up.
- Point to the exact repo surfaces that would need attention.
- Keep this as
recommended fix areas, not a redesign.
Reasoning Discipline
- Separate confirmed upstream changes from inferred risk.
- Base the main verdict on the latest stable release, not on prerelease or main-branch signals.
- If the repo docs are still compatible with upstream but the repository overstates shipped behavior or qualification scope, call that a repository truthfulness issue, not an upstream break.
- If the upstream docs are vague but the tagged stable source or shipped stable behavior is clear, cite the stable source and call out doc drift.
- Treat config precedence, profile resolution,
OPENAI_BASE_URL, sharedOPENAI_API_KEY,auth.json, credential pools,cron/jobs.json, and managed-install behavior as high-sensitivity by default. - Do not infer support for out-of-scope product changes that this repository explicitly rejects.
Output
Load references/report-template.md before writing the final answer.
The report should:
- cite the exact stable Hermes release audited and its publish date
- link the primary sources used
- separate confirmed upstream changes from inferred risk
- separate stable-verdict impact from prerelease or main-branch watchlist signals
- point to the exact repository surfaces that would break or need clarification
- include a short
recommended fix areassection only when the verdict iscompatible with caveatsorincompatible
Keep the output short, decisive, and evidence-backed.