CodeSearch Local Semantic Code Search

MANDATORY: QUERY CODESEARCH BEFORE GREP/GLOB

This is a PROTOCOL REQUIREMENT, not a suggestion. Failure to query CodeSearch first is a violation.

WHY THIS IS MANDATORY

• CodeSearch SQL: 0.002s | grep: 0.8s (400x slower)
• Agents using grep first waste tokens and time
• Index already exists at ~/.local/share/codesearch/index_v2.db

ALWAYS USE CODESEARCH FIRST

# Exact name lookup - 0.002s
sqlite3 ~/.local/share/codesearch/index_v2.db "SELECT file_path, line_number FROM entities WHERE name = 'MyFunction';"

# Fuzzy search - 0.004s
sqlite3 ~/.local/share/codesearch/index_v2.db "SELECT file_path, line_number FROM entities WHERE name LIKE '%Store%' LIMIT 10;"

# Semantic search
/codebase-search "authentication middleware pattern"

GREP IS ONLY ALLOWED WHEN:

• CodeSearch query returned zero results AND project confirmed not indexed
• Searching literal strings (error messages, comments, config values)
• Explicit user request for grep

FOR CONCEPTUAL QUESTIONS:

• "Where is X implemented?" → CodeSearch semantic search
• "Find similar patterns" → CodeSearch embeddings
• "How is feature Y built?" → CodeSearch first, then read files

Quick Commands

Semantic Search (V1 - Embeddings)

# Natural language search
/codebase-search "authentication middleware pattern"
/cfn-codesearch-search "error handling in API routes"

# CLI direct (global install preferred)
local-codesearch query "user login flow" --max-results 5

Structural Search (V2 - SQL on AST)

# Find all callers of a function
sqlite3 ~/.local/share/codesearch/index_v2.db \
  "SELECT * FROM refs WHERE target_name = 'MyFunction';"

# Find all functions in a file
sqlite3 ~/.local/share/codesearch/index_v2.db \
  "SELECT name, line_number FROM entities WHERE file_path LIKE '%myfile.rs' AND kind = 'function';"

# Find entities by project (multi-project isolation)
sqlite3 ~/.local/share/codesearch/index_v2.db \
  "SELECT COUNT(*) FROM entities WHERE project_root = '/path/to/project';"

Installation

Global install (recommended for multi-project use):

# From claude-flow-novice project
./scripts/install-codesearch-global.sh

# Verify
local-codesearch --version

This installs to ~/.local/bin/local-codesearch for access from any project.

Prerequisites

OPENAI_API_KEY is REQUIRED for indexing. Indexing will fail without a valid key.

# Option 1: Export before running
export OPENAI_API_KEY="sk-..."

# Option 2: Add to shell profile (~/.bashrc or ~/.zshrc)
echo 'export OPENAI_API_KEY="sk-..."' >> ~/.bashrc
source ~/.bashrc

# Option 3: Inline with command
OPENAI_API_KEY="sk-..." ./local-codesearch --project-dir /project index --path . --types rs,ts,py,sh,json,yaml,sql

Verify key is set:

echo $OPENAI_API_KEY  # Should show your key (not empty)

Index Management

# Index a project (first time or full rebuild)
# project_root is tagged via --project-dir; the inner `index --path .` walks that dir.
local-codesearch --project-dir /path/to/project index --path . --types rs,ts,tsx,js,jsx,py,sh,json,yaml,sql

# Incremental update (after code changes)
/codebase-reindex

# Check index stats
sqlite3 ~/.local/share/codesearch/index_v2.db "SELECT project_root, COUNT(*) FROM entities GROUP BY project_root;"

Indexing Internals

• .claude/worktrees (and worktrees, .worktrees) are excluded from indexing. They are ephemeral CFN agent git clones that duplicate source and nest node_modules; indexing them caused data collisions and memory blowup.
• Python (.py) is supported as a first-class indexed language, alongside rs/ts/tsx/js/jsx/sh/json/yaml/sql.
• No memory runaway: the post-commit indexer enqueues each commit's changes as a job file, then a single worker drains the queue under an flock. Concurrent commits never stack indexers; jobs are aggregated and deduped before indexing.
• Index stays fresh automatically via the post-commit hook .claude/hooks/cfn-post-commit-codesearch-index.sh (enqueue on every commit, single locked drain worker).

Key Features

• Multi-project isolation: Index multiple projects in single database without data collision
• Non-destructive: Indexing one project never deletes data from other projects
• Centralized storage: ~/.local/share/codesearch/index_v2.db
• Dual search: V1 semantic (embeddings) + V2 structural (SQL on AST)
• Fast: Rust binary with SQLite backend

Database Location

~/.local/share/codesearch/index_v2.db

For Agents (MANDATORY PROTOCOL)

DO NOT use grep/glob until you have queried CodeSearch. This is enforced.

# STEP 1: Query CodeSearch FIRST (required)
/codebase-search "relevant search terms" --top 5
# Or SQL:
sqlite3 ~/.local/share/codesearch/index_v2.db "SELECT file_path, line_number FROM entities WHERE name LIKE '%keyword%';"

# STEP 2: Query past errors/patterns
$HOME/.claude/skills/cfn-codesearch/query-agent-patterns.sh "description"

# STEP 3: Only if CodeSearch returns nothing, then use grep

Violation of this protocol wastes tokens and time. CodeSearch exists to prevent duplicated work.