Index processor on co-index's shared SocratiCode store (init-socraticode, STORE=external)

Goal

Index processor's codebase with SocratiCode against co-index's shared store: Qdrant at https://index.taild0fb76.ts.net:6333 and Ollama nomic-embed-text at index:11434. That is init-socraticode with STORE=external, as broker (broker#17), replicator and watcher did. The collections are created at the first index from the projectId (codebase_processor, context_processor, processor_symgraph_*). co-index keeps no project registry, and none of the recent adopters needed an issue in CannObserv/index.

Blocked on

Checked from co-processor on 2026-09-30:

#3 Vendor the cohort's agent skills #4 merged. It vendors init-socraticode (at c90e3c2) and adds the .worktrees/ and .claude/settings.local.json ignore rules.
Tailnet policy (operator). tailscale status lists only broker as a peer, and both 6333 and 11434 time out against index's IP, so this is policy, not DNS. Add tag:processor to the rule granting tag:index:6333,11434. Broker got the same one-line change, tracked on notifier#57 and applied in a broker#17 comment. The four-tag rule shown in CannObserv/index deploy/README.md is stale.
- Done 2026-10-01 (operator), verified from co-processor: index is a peer. Both index and index.taild0fb76.ts.net resolve to 100.65.25.88, ports 6333 and 11434 accept connections, Qdrant's TLS certificate verifies for the full name (/healthz 200), and Ollama answers on http://index:11434 (/api/version 200).
Qdrant key (operator). Pipe it from co-index into scripts/install_qdrant_key.sh, run from a read-only clone of CannObserv/index here (index docs/installing-the-key.md). It must print installed 64 chars. Never run it under bash -x.
Name resolution. This VM runs Tailscale with --accept-dns=false (AGENTS.md), and the reason isn't recorded. getent hosts index fails and index.taild0fb76.ts.net doesn't resolve. The certificate doesn't cover the short name, so QDRANT_URL needs the FQDN. Decision needed: tailscale set --accept-dns=true (observo#631's parity fix, which would also let AGENTS.md and DEPLOYMENT.md say broker instead of 100.97.91.19), or keep it off and use pinned host entries.
- Done in co-processor runs Tailscale with --accept-dns=false: move to cohort parity (observo#631) #8 / #8 co-processor runs Tailscale --accept-dns=true; tailscaled OOM drop-in; the broker by name #9 (2026-10-01): DNS on (CorpDNS: true), and the docs name broker. index and index.taild0fb76.ts.net resolve since the policy grant above.
Node on the session's PATH. SocratiCode launches through npx, and co-processor has no Node. How to install it is a separate decision.
systemctl disable --now docker.socket, as broker, watcher#310 and archiver did. Docker is installed and its socket is enabled. If the settings env block doesn't apply (an untrusted folder, or a server started before the block existed), SocratiCode silently falls back to starting local containers. A stray docker images on 2026-09-30 socket-activated docker.service, which was then stopped by hand.
- Done 2026-10-05 (operator, gate 1 of Contain the extraction child: it runs as the service user and can reach the service's secrets #2), verified 2026-10-06 at Contain the extraction child: it runs as the service user and can reach the service's secrets #2's install: systemctl is-enabled docker.socket says disabled, and is-active says inactive. The service user processor isn't in group docker, and the contained child can't open a socket at all (test_the_child_cannot_reach_the_hosts_sockets[/run/docker.sock], EPERM). Evidence: Contain the extraction child: it runs as the service user and can reach the service's secrets #2's closing comment.

Memory

Once processor.service is installed, the extraction child (RLIMIT_AS, oom_score_adj 1000) shares 7.7 GiB, no swap, with the SocratiCode server. So follow broker#33 and replicator#99:

Pin the server with a capped install (systemd-run --user --scope -p MemoryHigh=… -p MemoryMax=… choom -n 500 npm install --prefix ~/.socraticode/pin socraticode@<ver>), and set SOCRATICODE_SPEC to the same version. If sessions run from VS Code, also set it in claudeCode.environmentVariables (init-socraticode: the session-pin checks read SOCRATICODE_SPEC from their own environment — green while the session launches @latest gregoryfoster/skills#332).
Run the first index capped too. At 49 tracked files the index is cheap; the npm install is the expensive part.
Consider MemoryLow= for processor.service.

Decision needed: the version pin. The cohort pins 1.14.0 and observo is moving to 1.16.0. A client older than a collection's indexFormatVersion reads zero rows from it without an error (notifier#72), so pin forward, never back.

Repo side, after the blockers

Run init-socraticode with STORE=external, then:

Commit .socraticode.json (projectId: processor, relative linkedProjects for the siblings it reads: probably watcher, replicator, broker, archiver and notifier), plus one link stub per sibling.
.socraticodeignore: skills-vendor/ skills/ .claude/skills/ .skills/ .worktrees/ .claude/worktrees/ .wheelhouse/.
The context-artifacts manifest (without it the health hook reports nothing), and the health and reminder hooks, symlinked through install-hook.sh.
The six-variable env block in .claude/settings.json, in the same commit as .socraticode.json or after it.
A config test like broker's and archiver's: no QDRANT_COLLECTION_PREFIX, no SOCRATICODE_BRANCH_AWARE, and the key file confirmed ignored with git check-ignore.
Restart the session in a trusted folder before indexing, and confirm codebase_health reports Qdrant mode: external.

Pitfalls on record

A server that is already running never picks up a new env block (archiver#226).
A missing link stub, or a sibling collection that doesn't exist yet, is skipped without any warning, so a green includeLinked result doesn't prove every sibling answered.
Worktree sessions carry no key, and they resolve ../<sib> inside .worktrees/ (observo#654).
Only co-processor ever indexes processor: the lock is per host (D11).

#3 fix: CR 7 — SKILLS.md links the follow-ups to their issues (skills…

Moved to #2 (operator decision, 2026-10-05, at epic #24's check-in): the sudo systemctl disable --now docker.socket box. #2's host survey found the service user exedev in group docker with docker.socket active, which gives a compromised extraction child a path to root. #2 lands first and carries the test. When #5 is picked up, check the socket is off (systemctl is-enabled docker.socket → disabled) and tick the box by reference.

Hand-off from the burn-down epic #24, 2026-10-10. This comes after #45 and #47 (re-slotted 2026-10-10: #43 shipped; #45 is a deploy-blocking flake; #47 closes out the cutover after watcher#350), and it's the last unparked item. Since this brief: Watcher's soak passed and watcher#350 deleted its local extraction; nothing here depends on either. #39 shipped in #42.

Why now. It's agent tooling with no production effect, and every production-facing item ahead of it has shipped. Watcher's soak passed (watcher#326 closed 2026-10-09T22:07Z), so no heads-up is needed for anything here. And nothing here touches processor.service.

Operator decision (2026-10-10, at the check-in): pin socraticode@1.16.0, matching broker and observo. Watcher, replicator, archiver and notifier pin 1.14.0.

Known consequence: a 1.14.0 sibling that links codebase_processor reads zero rows from it without an error (notifier#72: a client older than a collection's indexFormatVersion sees nothing). No sibling links processor today. Say so in the PR.
Check, don't assume: that a 1.16.0 client reads the 1.14.0-format sibling collections processor will link (watcher, replicator, archiver, notifier) [stated, not verified]. Use codebase_search with includeLinked against each one, and check that results come back from each, not just "green" (the issue's pitfall: a silent skip looks green).

The body's blockers, re-checked on 2026-10-09T22:30Z:

#3 Vendor the cohort's agent skills #4 merged (2026-09-30), so init-socraticode is vendored (skills/init-socraticode). Tick it.
Node on PATH: node --version is v24.21.0 (Node.js 24 LTS on co-processor from NodeSource apt, versioned in deploy/ (agent tooling: mayfly, SocratiCode) #6). Tick it.
Tailnet and DNS: done (2026-10-01, co-processor runs Tailscale with --accept-dns=false: move to cohort parity (observo#631) #8/#8 co-processor runs Tailscale --accept-dns=true; tailscaled OOM drop-in; the broker by name #9).
docker.socket: done and ticked (2026-10-05, Contain the extraction child: it runs as the service user and can reach the service's secrets #2's gate 1).
The Qdrant key: still to do (operator). Nothing under ~/.socraticode, and no key file. The issue's step: pipe it from co-index into scripts/install_qdrant_key.sh, from a read-only clone of CannObserv/index (index docs/installing-the-key.md). It must print installed 64 chars. Never under bash -x.

What changed since filing:

The service no longer runs as exedev (Contain the extraction child: it runs as the service user and can reach the service's secrets #2). SocratiCode runs as exedev, the service as processor from /srv/processor/live. The service can't read exedev's home (ProtectHome=yes), the child can't open sockets, and SocratiCode can't touch the release. The body's memory section still holds: they share 7.7 GiB, with no swap.
processor.service is now installed and deployed by scripts/deploy.sh (Contain the extraction child: it runs as the service user and can reach the service's secrets #2, A drift signal when live lags origin/main (status#12's drift check) #35). If this item adds MemoryLow= (the body's "consider"), it goes in deploy/processor.service and ships through a deploy. That's a restart, and since Liveness check-in for processor run (co-status monitor co-processor-live) #39 a restart is a liveness blip: about 5 s, well inside the 900 s grace. Decide whether it's worth it. Today the service uses about 64 MiB (MemoryCurrent) under MemoryMax=6G.
broker#33 (closed 2026-10-06) is the closest prior art, at the same version: a capped npm install --prefix ~/.socraticode/pin socraticode@1.16.0. The repo's .claude/settings.json env block is enough even for VS Code-hosted sessions, and claudeCode.environmentVariables wasn't needed (its last comment). Follow its preflight wording to prove the running server is the pinned one.

Traps:

A server that's already running never picks up a new env block (archiver#226). Reload the window or restart the session after committing .claude/settings.json, then confirm with codebase_health (Qdrant mode: external) and the running server's command line (socraticode@1.16.0, not @latest).
Worktrees carry no key, and resolve ../<sib> inside .worktrees/ (observo#654). Index from the main checkout. .socraticodeignore must list .worktrees/ and .claude/worktrees/ (the body has both).
The skills SessionStart hook commits to main daily. Make sure no generated SocratiCode artifact lands in a tracked path it would commit: check git status after an index.

Gates (operator): the Qdrant key (above). The version is decided.

Done when: the body's "Repo side" list, plus:

codebase_health says Qdrant mode: external, and the server runs socraticode@1.16.0;
a search returns processor results, and includeLinked returns results from each linked sibling;
the config test (no QDRANT_COLLECTION_PREFIX, no SOCRATICODE_BRANCH_AWARE, key ignored by git check-ignore) passes in CI;
the memory decision (MemoryLow= or not) is recorded with its reason.

Ship: issue → branch in a worktree → PR (config, hooks, the config test, docs) → FF merge → git pull --ff-only → install the pin and index from the main checkout. A deploy only if MemoryLow= is added. The closing comment gives each box with its evidence, the known limits, and the follow-ups (or "none").

Kickoff prompt:

Pick up processor#5. Read its hand-off comment from epic #24 first.

Refresh from the burn-down epic #24, 2026-10-10T23:31Z. This is next now (#45 and #47 shipped; #1 closed). The hand-off above stands, re-checked just now:

Qdrant key: still not installed. There's no ~/.socraticode, so the operator's gate is unchanged.
Cohort pins are unchanged: broker and observo socraticode@1.16.0; watcher, replicator, archiver and notifier 1.14.0; status none. The operator's decision (1.16.0) stands.
Production state since the brief: Watcher no longer extracts (watcher#350), and Processor deploys through deploy.sh: gate a deploy on the commit's CI (status#11's CI gate) #34's gate with liveness on (Liveness check-in for processor run (co-status monitor co-processor-live) #39, test_liveness flakes: a check-in thread can still be alive just after its tick returns #45). If this item adds MemoryLow=, that deploy is about a 5 s liveness blip, inside the 900 s grace. No heads-up is needed any more.
Nothing else in the brief has moved.

Kickoff prompt:

Pick up processor#5. Read its hand-off comment from epic #24 first.

Shipped in #50 (FF b941173, CI green on main), installed and indexed from the main checkout on 2026-10-11. No deploy: nothing under src/ or deploy/ changed.

Blockers (body)

#3 Vendor the cohort's agent skills #4 merged: init-socraticode vendored (skills/init-socraticode).
Tailnet policy: done 2026-10-01 (above).
Qdrant key (operator): installed 2026-10-11 into ~/processor/.claude/settings.local.json: 64 chars, mode 600, ignored by the tracked .gitignore:22 (git check-ignore -v).
Name resolution: co-processor runs Tailscale with --accept-dns=false: move to cohort parity (observo#631) #8 / #8 co-processor runs Tailscale --accept-dns=true; tailscaled OOM drop-in; the broker by name #9.
Node on PATH: v24.21.0 (Node.js 24 LTS on co-processor from NodeSource apt, versioned in deploy/ (agent tooling: mayfly, SocratiCode) #6).
docker.socket: disabled (Contain the extraction child: it runs as the service user and can reach the service's secrets #2). Preflight: Docker not needed — the store is external and the embedder is not a container.

Repo side (body)

.socraticode.json: projectId: processor; linkedProjects ../archiver, ../broker, ../notifier, ../replicator, ../watcher, relative. Link stubs {"projectId": "<sibling>"} under /home/exedev/<sibling>/ (health-check: 5 configured, 5 resolved, 0 missing).
.socraticodeignore: the body's list (skills-vendor/ skills/ .claude/skills/ .skills/ .worktrees/ .claude/worktrees/ .wheelhouse) plus docs/plans/. Plans remain searchable as the implementation-plans context artifact. skills/brainstorming is the one non-symlink, and it's an upstream override, so skills/ goes whole.
Context manifest: 9 artifacts, validate-manifest OK, all 9 indexed (156 chunks).
Hooks: socraticode-reminder.sh (prefetch) and socraticode-health.sh, symlinked into skills-vendor/ by install-hook.sh, whose --check passes for both. Each is registered once.
The env block (six variables + SOCRATICODE_SPEC=socraticode@1.16.0), in the same commit as .socraticode.json.
Config test tests/test_socraticode_config.py: no QDRANT_COLLECTION_PREFIX, SOCRATICODE_BRANCH_AWARE, SOCRATICODE_PROJECT_ID or QDRANT_HOST in any env source; settings.local.json ignored by the tracked .gitignore; no key in the committed settings; the store never addressed without projectId: processor; the exact pin; the hooks; the index scope; the manifest. Passes in CI (#5 SocratiCode on co-index's shared store #50, and main at b941173). On co-processor all 33 pass with none skipped, including the driver-pin check.
Restarted in a trusted folder; codebase_health reports Qdrant mode: external (below).

Done when (hand-off)

Qdrant mode: external, and the server runs socraticode@1.16.0. After the window reload, the session's codebase_health: Qdrant mode: external, endpoint https://index.taild0fb76.ts.net:6333 reachable, Ollama http://index:11434 with nomic-embed-text available. Process table: npm exec socraticode@1.16.0 is a child of the session's claude (extension 2.1.292). The env block alone was enough here, as on broker#33; claudeCode.environmentVariables wasn't needed. The driver's pre-install ~/.socraticode/pin is 1.16.0, installed capped (systemd-run --user --scope -p MemoryHigh=1200M -p MemoryMax=1536M choom -n 500), and the npx cache was filled for the exact spec.
Processor search, and includeLinked results from each sibling. First index (capped, via the driver): completed with no failure, graph READY, artifacts 9/9. codebase_status: codebase_processor green with 600 chunks; graph 54 files, 80 edges. health-check --probe src/processor/__main__.py: verdict ok, built by 1.16.0. That keeps the AGENTS.md block on variant A; the 60.8% unresolved share is a statistic, not a defect. includeLinked hits per project, counted rather than read as a green light:
- 6 queries through the pinned server: watcher 51, archiver 49, replicator 42, processor 17, broker 13, notifier 8.
- Native, from the restarted session: watcher 12, replicator 11, archiver 8, broker 5, processor 4. A notifier-targeted query then returned notifier's dispatcher.py, html_render.py, constants.py and test_dispatcher.py.
- So a 1.16.0 client reads the 1.14.0-format collections (watcher, replicator, archiver, notifier). The [stated, not verified] claim is now verified.
Config test passes in CI: above.
Memory decision: no MemoryLow=. Preflight: cgroup2 is mounted without memory_recursiveprot and system.slice grants none, so a MemoryLow= would be inert without a host system.slice drop-in as well. The threat is OOM, not reclaim: processor.service already sits at OOMScoreAdjust=-500 (~64 MiB resident under MemoryMax=6G), and each extraction child volunteers at +1000. The pin removes the npm-at-session-start peak behind broker's 2026-09-16 outage. Recorded in docs/SOCRATICODE.md and DEPLOYMENT.md's Node section; revisit if the service's resident memory reaches hundreds of MiB.
Trap 3: git status stayed clean after the index; no generated artifact sits in a tracked path.

Known limits

A 1.14.0 client (watcher, replicator, archiver, notifier) that links codebase_processor reads zero rows from it without an error (notifier#72). None links processor today; one that wants to must move to 1.16.0 first.
codebase_processor is written from co-processor's main checkout only. Worktrees carry no key and resolve ../<sibling> inside .worktrees/ (observo#654).

Follow-ups

None.

Navigation Menu

Repository navigation

Index processor on co-index's shared SocratiCode store (init-socraticode, STORE=external) #5

Goal

Blocked on

Memory

Repo side, after the blockers

Pitfalls on record

Blockers (body)

Repo side (body)

Done when (hand-off)

Known limits

Follow-ups

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Index processor on co-index's shared SocratiCode store (init-socraticode, STORE=external) #5

Description

Goal

Blocked on

Memory

Repo side, after the blockers

Pitfalls on record

Activity

gregoryfoster commented on Oct 5, 2026

gregoryfoster commented on Oct 10, 2026

gregoryfoster commented on Oct 10, 2026

gregoryfoster commented on Oct 11, 2026

Blockers (body)

Repo side (body)

Done when (hand-off)

Known limits

Follow-ups

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions