open-code-review-setup
Set up Alibaba Open Code Review (OCR) on a repository that lacks it, or audit and update an existing setup. Use when creating a new repository, onboarding an existing repo to automated AI code review,
SKILL.mdskills/engineering/open-code-review-setup/SKILL.md
GitHubOpen Code Review Setup
Lifecycle operation: detect → install → configure → verify. For PR metadata, agent labels, and the token-spend semantics of rule.json, use ai-agent-pr-metadata — this skill only installs and validates the plumbing.
1. Detect
ls .opencodereview/ .github/workflows/ 2>/dev/null | grep -i -E 'ocr|open-code-review'
gh variable list --repo <owner>/<repo> | grep OCR_
gh secret list --repo <owner>/<repo> | grep OCR_LLM_AUTH_TOKEN
- All present → audit only: confirm the action is pinned to a SHA, the maintainer gate matches the repo owner, and the model var is still wanted. Do not clobber local customizations.
- Anything missing → install/configure the missing pieces below.
2. Install
Branch per git workflow (feature/issue-N-open-code-review, PR to the default integration branch — never commit to main/staging directly).
- Copy
templates/open-code-review.ymlandtemplates/ocr-manual-review.ymlinto.github/workflows/. - Set the maintainer gate in both files (
github.event.pull_request.user.login == '<maintainer>'/github.event.comment.user.login == '<maintainer>') to the account that opens PRs (the bot account whose token creates PRs, if agents open them). - Copy
templates/rule.jsonto.opencodereview/rule.jsonand adaptexcludeto the stack:- JS/TS:
node_modules/**,dist/**,pnpm-lock.yaml,package-lock.json - PHP/Laravel:
vendor/**,storage/**, decide explicitly ontests/**(built-in excludes do not cover*Test.php) - Static/binary assets (images, fonts, brand files): exclude their directories
- Keep
**/*.md,**/*.json,**/*.lockunless docs/schema review is wanted
- JS/TS:
- Pin
alibaba/open-code-reviewto the current release SHA (never a floating tag).
3. Configure
Repo variables (non-secret):
gh variable set OCR_LLM_URL --repo <owner>/<repo> --body "https://openrouter.ai/api/v1"
gh variable set OCR_LLM_MODEL_FREE --repo <owner>/<repo> --body "nvidia/nemotron-3-super-120b-a12b:free"
gh variable set OCR_LLM_MODEL_FALLBACK --repo <owner>/<repo> --body "<paid-model-id>"
gh variable set OCR_USE_ANTHROPIC --repo <owner>/<repo> --body "false"
- Default free model:
nvidia/nemotron-3-super-120b-a12b:free(OpenRouter free tier, 262k ctx). Override per repo only when asked; verify any model ID againsthttps://openrouter.ai/api/v1/modelsfirst. - Fallback model: use a known-working paid model already proven for this deployment context if one exists, otherwise a cheap general-purpose paid model — verify against
https://openrouter.ai/api/v1/modelsfirst, same as the free model. The preflight probesOCR_LLM_MODEL_FREEfirst and falls back toOCR_LLM_MODEL_FALLBACKon any non-2xx response (quota exhaustion, outage, etc.). - Secret:
OCR_LLM_AUTH_TOKEN= OpenRouter key. Pipe it from the local secrets store straight intogh secret set— never print, log, or commit the value:
grep -m1 '^OPENROUTER_API_KEY=' <env-file> | cut -d= -f2- | tr -d '\r' | gh secret set OCR_LLM_AUTH_TOKEN --repo <owner>/<repo>
- No key available, or preflight returns 401 → stop and use
hitl-blocker; do not retry automation.
4. Verify
- Open the setup PR. The
pull_requesttrigger runs the workflow from the PR head, so the PR reviews itself — watchgh run list. - Preflight must pass (HTTP 2xx). 401 = bad key (→ step 3 HITL); other codes = check
OCR_LLM_URL/model ID. - Confirm the review comment lands on the PR and ends with the
Review metadatafooter perai-agent-pr-metadata. - Compare
tool_calls/input_tokensin the workflow log's=== OCR result ===block after tuningrule.jsonto confirm spend dropped. ocr rules check --rule .opencodereview/rule.json <path>verifies a path matches before relying on it (no LLM call).- Confirm both probe branches are exercised at least once: the free-model probe should succeed under normal conditions; to test the fallback branch deliberately, temporarily set
OCR_LLM_MODEL_FREEto an invalid model ID viagh variable setin a throwaway test, confirm the workflow logs "Free model unavailable ... falling back to paid model", then revert the variable to its real value. - The preflight probe only proves the model is reachable (1-token reply) — it does not prove a real review will finish within
llm_timeout. A free-tier model can pass the probe and still time out mid-review on a large diff, and since fallback only triggers on a failed probe (not a failed review), a slow-but-reachable free model gets no fallback.llm_timeout: '300'is the template default and a starting point, not a guarantee — raise it further per repo if reviews are timing out on large diffs.
Guardrails
- Never print, commit, or comment secret values; public artifacts must use
Credential details: [redacted]. - The manual
/ocr-reviewworkflow must stay gated to the maintainer — an open comment trigger lets anyone spend LLM budget. - Never check out or execute PR-supplied code in OCR workflows; the action diffs content remotely.
- Existing
rule.jsonexclusions are deliberate cost controls — treat removals as spend increases, not cleanups.