omcustom:agora
Multi-LLM adversarial consensus loop — 3+ LLMs compete to find flaws in designs/specs until unanimous agreement is reached
SKILL.md.claude/skills/agora/SKILL.md
GitHubAgora: Multi-LLM Adversarial Consensus
3개 이상의 LLM(Claude, Codex/GPT, Gemini)이 경쟁적으로 설계/문서의 결함을 찾고, 만장일치 합의에 도달할 때까지 반복하는 적대적 교차 검증 스킬.
Prerequisites
codex-execskill (Codex/GPT 호출)gemini-execskill (Gemini 호출)- Agent Teams enabled (
CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1) or Agent tool available
Usage
/agora docs/design.md # Default: 3 LLMs, unlimited rounds
/agora docs/design.md --rounds 10 # Max 10 rounds
/agora docs/design.md --severity-threshold HIGH # Exit when no HIGH+ findings
/agora docs/design.md --models claude,codex # 2 LLMs only
Workflow
Phase 1: Setup
- Read the target document
- Create Agent Team:
TeamCreate("agora-review") - Create review tasks per focus area
Phase 2: Spawn Reviewers (parallel)
Spawn 3 reviewers as Agent Team members:
Agent(name: "claude-critic", model: opus, effort: max)
→ 20-point deep adversarial review
Agent(name: "codex-critic", model: opus)
→ Invoke Skill(codex-exec) for GPT perspective + independent Claude analysis
Agent(name: "gemini-critic", model: opus)
→ Invoke Skill(gemini-exec) for Gemini perspective + independent Claude analysis
Phase 3: Independent Review
Each reviewer performs adversarial review with this template:
For EACH review point:
### Round N: [Topic]
**Severity**: CRITICAL / HIGH / MEDIUM / LOW
**Flaw**: [Specific, concrete problem description]
**Evidence**: [Why this is real, not theoretical]
**Impact**: [What happens if not addressed]
**Counter-argument**: [Best case FOR the current design]
**Verdict**: KEEP / MODIFY / REJECT
Review areas (adapt to document type):
- Architecture fundamentals
- Component/service design
- Data architecture
- Security & resilience
- Feasibility & deployment
- Testing strategy
- Operational complexity
Phase 4: Cross-Review (Peer-to-Peer)
Each reviewer sends findings to the other two via SendMessage.
Counter-review template:
- Which findings do you AGREE with? (and why)
- Which findings do you DISAGREE with? (evidence-based rebuttal)
- What did they MISS that you caught?
- What did they catch that you MISSED?
- SEVERITY adjustments — upgrade or downgrade with justification
Phase 5: Synthesis
Team lead aggregates all findings:
UNANIMOUS CRITICAL: [findings all 3 agreed on]
STRONG AGREEMENT: [findings 2/3 agreed on]
SPLIT DECISIONS: [findings with disagreement + resolution]
Determine verdict:
- BUILD: No CRITICAL, no unresolved HIGH
- BUILD WITH CHANGES: No CRITICAL, HIGH findings have accepted mitigations
- REDESIGN: Any unresolved CRITICAL findings
- ABANDON: Fundamental concept is flawed
Phase 6: Loop (if REDESIGN)
- Team lead produces/delegates redesign addressing ALL critical findings
- New version sent to ALL reviewers:
SendMessage(to: "*") - Reviewers re-review → GOTO Phase 4
- Repeat until EXIT criteria met
Phase 7: Exit (consensus reached)
When ALL reviewers agree BUILD or BUILD WITH CHANGES:
- Produce final consensus report
- Write to
.claude/outputs/sessions/{date}/agora-{topic}-{time}.md
Tool: Writing artifacts under .claude/outputs/
CC sensitive-path check inspects tool target paths and triggers permission prompts on .claude/ regardless of bypassPermissions and allow rules (refs: #960, #961, #978, #981, #1016).
To write agora results under .claude/outputs/sessions/:
- Write the artifact body to
/tmp/agora-$(date +%H%M%S).mdfirst (Write tool target =/tmp, no sensitive-path trigger) - Use a
/tmp/*.shBash script to move/copy the file under.claude/outputs/sessions/$(date +%Y-%m-%d)/(Bash target =/tmp, script-internalcpto.claude/is not audited) - Read-only Bash on
.claude/outputs/(e.g.,cat,head,wc) is allowed for verification
Reference: feedback_sensitive_path_tmp_bypass.md, R006 sensitive-path handling, #1016, #1045.
- Shut down team:
SendMessage(to: "*", message: {type: "shutdown_request"})
Reviewer Principles
- NEUTRAL — no reviewer has home team advantage
- COMPETITIVE — find flaws others missed
- CRITICAL — "fewer than 5 CRITICAL flaws = not looking hard enough"
- EVIDENCE-BASED — every finding cites specific evidence
- CONSTRUCTIVE — every flaw includes recommended fix
- CONVERGENT — goal is consensus, not endless disagreement
Consensus Criteria
| Condition | Required |
|---|---|
| CRITICAL findings resolved | ALL |
| HIGH findings resolved or accepted | ALL |
| All reviewers rate BUILD or BUILD WITH CHANGES | YES |
| Cross-review disagreements resolved | ALL |
Output Format
# Agora Consensus Report
## Document: [path]
## Rounds: [N]
## Reviewers: [list with LLM models used]
## Verdict: [BUILD / BUILD WITH CHANGES / REDESIGN]
## Unanimous Findings
| # | Finding | Severity | All 3 Agree |
|---|---------|----------|-------------|
## Required Changes Before Build
1. [change with source reviewer]
2. ...
## Accepted Risks
- [finding accepted with justification]
## Unique Contributions Per Reviewer
| Reviewer | Findings Others Missed |
|----------|----------------------|
## Process Metrics
- Rounds: N
- Total findings: N
- Cross-adopted: N
- Severity upgrades: N
- Severity downgrades: N
- Disagreements raised: N
- Disagreements resolved: N/N
Configuration
# Default settings
agora:
max_rounds: unlimited # Set --rounds to limit
severity_threshold: HIGH # EXIT when no findings >= threshold
models:
- claude (opus, max effort)
- codex (via codex-exec skill)
- gemini (via gemini-exec skill)
review_points: 20 # Per reviewer
cross_review: true # Peer-to-peer sharing
auto_redesign: true # Auto-produce redesign on REDESIGN verdict
Anti-Patterns
| Anti-Pattern | Why Wrong | Correct |
|---|---|---|
| Single LLM review | Misses blind spots | 3+ LLMs find complementary flaws |
| No cross-review | Reviewers don't challenge each other | Peer-to-peer sharing surfaces disagreements |
| Accepting first BUILD | May miss edge cases | Loop until ALL agree |
| Ignoring split decisions | Unresolved disagreements fester | Resolve every split with evidence |
| Push for consensus too fast | Premature agreement | Let reviewers challenge freely |
When spawning agents via the Agent tool during this skill's execution, always pass mode: "bypassPermissions". The Agent tool default (acceptEdits) overrides agent frontmatter permissionMode, causing permission prompts during unattended execution.
Ontology Convergence (PoC)
Source: #993 (from ouroboros #966 re-evaluation, Option C deferred → PoC 섹션으로 내재화) Status: Experimental — default disabled
agora는 기본적으로 만장일치 기반으로 종료하지만, 의미적 유사도 기반 조기 종료를 PoC로 지원합니다.
Rationale
여러 라운드 후 모든 에이전트의 마지막 응답이 의미상 거의 동일하면(semantic similarity ≥ threshold), 만장일치를 기다리지 않고 조기 수렴으로 판단하여 토큰 비용을 절감합니다.
Configuration
ontology_convergence:
enabled: false # 기본 비활성 (PoC)
threshold: 0.95 # cosine similarity 최소값
min_rounds: 2 # 최소 라운드 (너무 이른 종료 방지)
embedding_model: voyage-3.5 # 또는 openai-text-embedding-3
Algorithm
- 각 라운드 종료 시 participant 응답의 embedding 계산
- Pairwise cosine similarity 매트릭스 생성
- 최소 유사도(min pairwise similarity) 계산
min_sim ≥ thresholdANDrounds ≥ min_rounds→ 조기 종료
Trade-offs
| 장점 | 단점 |
|---|---|
| 토큰 절감 (수렴 시 2-3 라운드 단축) | embedding 계산 오버헤드 |
| 만장일치 편향 완화 (의미 일치만으로 충분) | threshold 튜닝 필요 (프로젝트마다 다름) |
| 정량적 수렴 지표 | 오분류 시 조기 종료 리스크 |
Activation
현재 PoC 단계. 활성화 시 agora 스킬 호출 파라미터에 --ontology-convergence=true 추가. 프로덕션 승격 결정은 3개월 후 데이터 기반 재평가 (연계: #992 PAL Router Defer+observe 전략과 동일 원칙).
Cross-references
- #993 (source)
- #966 ouroboros 재평가
- guides/agent-design/pal-cost-routing-analysis.md (유사한 Defer+observe 전략)
Anti-Groupthink Mode (Optional)
agora의 기본 워크플로우는 만장일치 수렴(convergence)이 목표지만, 토론 과정에서 집단사고(Groupthink) 위험이 있을 때 anti-groupthink mode를 활성화할 수 있습니다.
Activation
스킬 호출 시 인자로 활성화:
/agora docs/design.md --mode anti-groupthink
Mechanisms
| 메커니즘 | 동작 |
|---|---|
| Devil's Advocate slot | 리뷰어 1명이 전담 반대자 역할 — 합의 형성 시도에 항상 반대 입장 견지 |
| Minority opinion protection | 1명만 주장하는 의견도 보존, 기각 시 명시적 정당화(3개 근거) 필수 |
| Round soft cap | 라운드 3회 도달 시 합의 미도달 영역은 "합의 없음 — 분기 결정 필요"로 종결 (기본 워크플로우는 무한 루프 가능) |
Reviewer Role Adjustment
기본 모드(3 reviewers)에 anti-groupthink mode 적용 시:
claude-critic→ Devil's Advocate 전담 (모든 합의 시도에 반대 입장)codex-critic,gemini-critic→ 일반 리뷰 (변경 없음)
Round soft cap이 작동하면 최종 보고서에 "UNRESOLVED — BRANCHING DECISION NEEDED" 섹션이 추가됩니다.
When to Use Anti-Groupthink Mode vs roundtable-debate
| 상황 | 권장 스킬 |
|---|---|
| 합의가 필요하지만 위험 발굴도 필요 | agora --mode anti-groupthink |
| 합의 자체가 불필요, 다양한 시각이 산출물 | roundtable-debate |
| 단순 검증 (통과/실패) | agora (기본 모드) |
자세한 비교는 guides/multi-agent-debate-patterns/ 가이드 참조 (별도 wave에서 생성 예정).
Attribution
Devil's Advocate + minority protection 메커니즘은 cc-roundtable 패턴에서 차용되었습니다. (roundtable-debate 스킬과 공유 메커니즘)