memory-search-advanced
Use when running thrum memory search with RAG (--near), composing complex predicate stacks, paginating large result sets, interpreting MEM-001..010 hint codes, or diagnosing embedding adapter behavior
SKILL.mdclaude-plugin/skills/memory-search-advanced/SKILL.md
GitHubmemory-search-advanced — RAG, hint codes, complex predicates
Load this sub-skill from thrum-memory (umbrella) for predicate needs beyond
tag + kind + recency filtering.
Predicate surface (shipped)
thrum memory search combines all predicates with AND:
Query:
positional args — FTS query text; `thrum memory search "<topic>"` is
equivalent to --grep "<topic>" (multiple args join with single spaces;
giving both positional and --grep is refused as ambiguous)
Identity / structure:
--kind <kind>, --tag <tag> (SINGLE tag per query), --scope project|agent|role,
--status <status> ("*" for any; default active)
Text:
--grep <terms> (full-text MATCH over title + body_oneline/short/full;
terms matched literally — punctuation is not FTS5 operators)
Semantic (when embedding adapter enabled):
--near "<text>"
Time:
--since 7d | <Go duration> | <RFC3339>
Sort & limit:
--sort created_at_desc|updated_at_desc|relevance|pinned_first
--limit 10 (default)
--offset N
Output formats: table (default), ids, oneline, short, full, json.
Tag-set OR is NOT expressible in one query — --tag takes a single tag. For OR
semantics, run one search per tag and merge.
DESIGNED — NOT YET SHIPPED (do not use)
The original memory-search design specifies a wider predicate surface that the
shipped CLI does not implement. The flags below DO NOT EXIST today — the CLI
rejects them with unknown flag. They are recorded here as the spec diff for
the eventual search-surface implementer, NOT as usable commands:
--id, --kinds (multi-kind), --subkind, --agent / --author, --pinned,
--tags-any (tag-set OR),
--since-created, --since-updated, --until,
--parent <id>, --supersedes <id>, --superseded-by <id>,
--related-to <id>, --derived-from <id>,
--grep-field <oneline|short|full|title>, --regex,
--near-id <id>, --near-limit N
Workarounds with the shipped surface: time filtering = --since only (no
created-vs-updated split); per-field grep = not available (grep spans title +
all three zooms); edge predicates = not available (edge reads are the
memory.listByEdge RPC, no CLI); --pinned = --tag pinned (pinned is a
tag).
Pagination patterns
Default --limit 10 keeps context bounded. Walk a large set:
thrum memory search --kind session_summary --since 30d --limit 25 --offset 0
thrum memory search --kind session_summary --since 30d --limit 25 --offset 25
For counting: thrum memory count is the cheap O(1) counter, but it counts by
--agent / --role only — it accepts NO kind/time filters. To count a filtered
set, read the total field from the JSON response (a separate COUNT query,
computed before LIMIT applies):
thrum memory count --role researcher # cheap, coarse
thrum memory search --kind session_summary --since 30d \
--limit 1 --format json | jq '.total' # filtered count
Do NOT count via --limit 0 --format ids | wc -l — the --limit 0
streaming/export reservation is currently non-functional (the CLI drops
limit=0 from the RPC params and the daemon clamps the absent limit to 10, so
output caps at 10 rows regardless of the real match count).
RAG semantics (--near)
--near "<text>" runs semantic search against memory embeddings. Three
per-memory vectors are stored — one per zoom level, embedding the title-prefixed
string:
<title> — <body_oneline><title> — <body_short><title> — <body_full>
Ranking is score = max(sim_oneline, sim_short, sim_full). Each result carries
matched_zoom so the CLI can render the zoom level that actually matched.
Per-zoom embeddings let topical queries dock against oneline while
deep-content queries dock against full.
Length-bias caveat — read before trusting --near results
Title-prefixed oneline strings (~40-50 chars) and body_full (up to 256 KB)
embed in different subspaces of granite-30m-english's 384-dim space. Short
queries can systematically match oneline over full for length-similarity
reasons rather than semantic-relevance reasons. This is a real bias inherited
from the embedding model.
Detection and recovery (manual in v1; a future MEM-011+ zoom-divergence detector is reserved for this case; the shipped MEM-008 is the embed-enqueue-failed warning, not a zoom-divergence detector):
- Inspect
matched_zoomin each result. If short queries return rows dominated bymatched_zoom=oneline, the ranking may have been misled. - Re-run with a longer query phrasing.
- Add
--kind <X>to narrow before semantic re-rank. - Inspect the full body manually, following the shared 3-step read pattern
(
thrum-memory): triage withthrum memory show <id> --zoom shortfirst, then fetch withthrum memory show <id> --zoom fullonce confirmed.
Mitigations weighed and deferred (don't reach for them yet): weighted
combination, cross-encoder re-rank, per-kind length normalization. v1 ships
max-of-three; tune later only if matched_zoom diagnostics surface systematic
problems.
Hint codes (MEM-001..010)
Hints render as a hints array in --json mode and as inline notices in human
mode. Suppress everywhere with THRUM_NO_HINTS=1. Severity is warn for
blocking-shaped advice, info for nudges.
| Code | Trigger | What it tells you |
|---|---|---|
MEM-001 |
search returned >50 matches | Narrow with --kind / --tag / --since |
MEM-002 |
--near supplied without --kind |
Add --kind to narrow before semantic re-rank (deferred) |
MEM-003 |
empty result with two or more narrowing predicates | Drop the most-restrictive flag |
MEM-004 |
--near with embedding adapter disabled |
Hard refusal — enable the adapter (see setup below) |
MEM-005 |
--grep matched nothing AND field is empty for most rows of kind |
Broaden: drop --kind |
MEM-006 |
--near found rows with embed_status=pending → FTS fallback |
Wait for backfill or run memory.embed.rebuild (deferred) |
MEM-007 |
--near skipped rows whose embedding_model ≠ configured model |
Run backfill to migrate vectors (deferred) |
MEM-008 |
memory.create enqueue to embed worker failed |
Best-effort; the worker will retry via memory.embed.rebuild |
MEM-009 |
memory.edit projection detected concurrent edit; LWW loser |
Loser's value is preserved in memory.history (deferred surfacing) |
MEM-010 |
memory.create omitted body_short or body_full |
Three-zoom retrieval works better with all three; non-blocking nudge |
Deferred triggers: MEM-002, MEM-006, MEM-007 are registered today but the
emission path lights up in a future epic (--near adapter wiring +
multi-model support). MEM-009 emission lights up when LWW-loser surfacing
lands in the projection response. The codes are stable; only the trigger
conditions are in flight.
Hard refusal: MEM-004 is the only hint with AllowForce=false. There is no
recovery flag — the adapter is either present or absent.
Embedding adapter awareness
--near only works when the embedding adapter is enabled:
thrum config set llm.embedding.enabled true
thrum config set llm.embedding.provider ollama --force
thrum config set llm.embedding.base_url http://localhost:11434 --force
thrum config set llm.embedding.model "granite-embedding:30m-english"
Embedding work is queued on write (OS-nice +10, single-flight per memory_id,
exponential backoff). Writes never block on embedding. Searches degrade
gracefully to FTS for not-yet-embedded rows and surface MEM-006 when the
degradation happens.
Edits mark only affected zoom-levels as embed_status=stale; tag-only edits
re-embed nothing, title-only edits re-embed all three. Old vectors stay
queryable during the staleness window.
Cold-start backfill runs once on the first daemon boot with the adapter enabled.
Disable with embedding.backfill_on_boot: false; rate-limit via
embedding.max_per_minute (default 60).
Composing complex predicates
Stack predicates left-to-right; they AND. Use --scope only when you
specifically want to filter by assembly scope (it is NOT visibility — see
umbrella skill).
Example: every active arch decision tagged substrate, edited recently,
semantically related to "RPC routing" (tag-set OR = one query per tag):
thrum memory search \
--kind arch_decision \
--tag substrate \
--status active \
--since 14d \
--near "RPC routing" \
--limit 25
Example: every superseded session summary that touched a given epic, paged:
thrum memory search \
--kind session_summary \
--status superseded \
--tag epic-thrum-abc \
--sort updated_at_desc \
--limit 50 --offset 0