[roodboi]

Summary

Fix unit G111: the three HACK-1463 follow-ups from the #497 verifier. One of them was not a test problem. The carrier-state case failed under load because the storage transport refused healthy commands, so this PR includes one small product fix in groupAbsent.

Root causes and fixes

(1) Carrier-state call-site case "saw 9 inspects": a real refusal under load

The count was a symptom. I ran the three carrier-state cases in a loop at load average 100-240 with diagnostics on every refusal path. About 1 run in 6 failed. Each time, the carrier had refused before the fault fired, at whichever recorded command was running (storage-helper-create or storage-helper-start), as unclassified. One failure stopped after inspection 5, another before inspection 1. The inspect count was simply wherever that refusal landed.

The diagnostics traced the refusal to groupAbsent in src/lib/native-compose-storage-witness-docker.ts. A recorded command makes host identity reads (/bin/ps, sysctl kern.bootsessionuuid) through the unrecorded runner. After each read exits, the runner proves its process group is gone with kill(-pgid, 0). Only ESRCH was accepted, and any other error refused at once, with no cause attached. On macOS, kill(-pgid, 0) can return EPERM just after the group's last member exits. A standalone probe (spawn a detached ps, await its exit, probe the group) saw EPERM once in 400 exits on this host. So a healthy read was refused, and the refusal surfaced as an unclassified carrier refusal at the enclosing stage.

Fix: EPERM now means "not yet absent" (the group still has a member this process cannot signal). The wait continues to the same monotonic deadline: 3 s for an unrecorded command, or the 15 s settlement budget. Only ESRCH proves absence. If EPERM persists, the command still refuses at its deadline as probe-budget. Any other error still refuses at once. This follows the existing group-absence polls in native-compose-file-command.ts and native-host-hook-runner.ts. The carrier reference doc said "EPERM/EIO ... refuse". It now says that EPERM is not absence and is waited out to the same budget.

Test change: the carrier-state cases now assert the reported refusal (stage and reason) before the inspection count. A refusal that happens before the fault then fails with its own stage and reason instead of a bare count. Only the fault reports helper-policy-stability, and it refuses at once. So once that refusal is confirmed, the count still pins which inspection reached the tagged call site, and the property the case proves is kept.

New regression tests in tests/native-compose-storage-witness-docker-io.test.ts drive the real unrecorded runner with a process.kill spy that answers EPERM to the group probe:

• A group that answers EPERM three times and then ESRCH completes (exitCode 0, exact stdout).
• A group that always answers EPERM is never treated as absent and refuses as probe-budget at its 3 s budget.

(2) Probe-budget recorded slow start: wall time is host work, not the 20 s delay

The issue assumed the simulated 20 s start used 20 s of the 30 s Bun budget. It does not. The engine stubs Date.now and advances it on a virtual clock (engine.elapsed), so SLOW_START_MS costs no wall time. The wall time is the recorded branch's host work: three published commands, each with stopped-child ps/sysctl identity reads, private-state publications and settlement, plus finish() retirement. At load 120-140 that took more than 30 s. Shortening the simulated delay would not change it.

Fix: the recorded case gets a named budget, RECORDED_PROOF_CASE_BUDGET_MS = 120_000, documented as a hang guard rather than a speed assertion. SLOW_START_MS stays at 20 s, so the "slower than the former 15 s cap" property and its mutation proof are unchanged. Biome re-wraps that one test body because its last argument is now an identifier.

(3) Finalizer owner-confirmed refusal tagged

The deferred finish's if (!nativeComposeStorageCommandOwnerConfirmed(owner)) return refuse(); now calls refuseObserved("storage-helper-cleanup", "private-state"). The finalizer is registered only after all three commands settled. Only a running command can invalidate the owner, so no deadline can veto it at this point. A vetoed owner is the invocation's saved private state, reported at the carrier's own cleanup stage, as the binding check right after it is. private-state is outside deadlineReasons, so it never reads storage-budget past the deadline. Both reference docs say so.

The new carrier case drives the real carrier against the synthetic daemon. It captures the real owner from createNativeComposeStorageCommandOwner and invalidates it with invalidateNativeComposeStorageCommandOwner after the invoke returns and before finish(). It asserts {stage: storage-helper-cleanup, reason: private-state}, a late reason of private-state, and refusal before the finish's binding read (48 reads, not 49) and before evidence retirement.

Mutation proof

Each mutant was applied to the source, then its focused test was run with --timeout 60000. The source was restored byte for byte afterwards (checked by comparing file contents).

Mutant	Killed by	Observed
Finalizer owner check back to return refuse()	new vetoed-owner carrier case	refusal undefined, late storage-budget
groupAbsent refuses EPERM at once (the old code)	both EPERM io cases	first: refused; second: unclassified instead of probe-budget
groupAbsent treats EPERM as absent	both EPERM io cases	first: answered 1 instead of 3; second: resolved instead of refusing
Fixed 15 s cap restored for command probes	recorded exec/up slow-start case	proof refused
Remove-spawned state check back to untagged if (!same(...)) return refuse()	carrier-state row "once its remove child is spawned"	storage-helper-cleanup / unclassified, late storage-budget

Evidence

• Bun 1.4.2 (mise x bun@1.4.2), bun install --frozen-lockfile; no bun.lock change.
• Root bun run check: privacy-check ok, 34 warnings, all of them complexity warnings that were already there (invoke is still 57). bun run --cwd packages/cli typecheck and root bun run typecheck: 0 errors. ultracite check on the three changed test files: clean.
• gitleaks 8.30.1 (gitleaks dir over the git log -p origin/next..HEAD export): no leaks.
• Focused files at the PR head, run serially with --timeout 60000 at load average about 100: docker IO 15 pass, probe budget 6 pass, command record 34 pass, docker witness 90 pass, carrier call sites 18 pass (767 s).
• Before the fix: the carrier-state cases failed 5 times in about 16 looped runs at load average 100-240 (shared host, some runs with an 18-24 worker CPU generator). Every failure was the pre-fault unclassified refusal described above.
• 30 runs under load at the PR head c487960. Each loop ran a yes CPU generator that a shell trap stopped on exit, and every run was a fresh bun test --timeout 60000 process:
  • (1) the three carrier-state cases ("stopped helper carrier …"): 30/30 runs pass (90/90 cases) with 18 generator workers at load average 70-126. Each run took 100-245 s.
  • (2) the recorded exec/up slow-start case: 30/30 pass with 18 workers at load 44-72 (4-8 s per process), and 30/30 pass with 120 workers at load 112-321. Under the heavier load each whole bun test process took 50-71 s, more than the former 30 s case budget.
• Full suite: hosted CI run 40143860376 at the exact PR head c487960 is all green: test shards 1-4 and aggregate, docker-e2e shards 1-4 and aggregate, Candidate core (Linux, macOS), Config compiler (ubuntu-latest, ubuntu-24.04-arm, macos-latest, macos-15-intel), Runtime state models, linux-process-lifetime, runtime-images and secret-scan.

Gaps

• The EPERM root cause is shown by a standalone probe and by diagnostics on the refusal path. The diagnostics were temporary and are not committed. I did not establish exactly which macOS kernel state produces EPERM. The doc comment says only what was observed.
• The io regression tests use a process.kill spy, so they prove the transport's handling of EPERM, not that the kernel produces it.
• Other group-absence checks in the repository were not changed. Some already retry any non-ESRCH result (native-compose-file-command.ts, native-host-hook-runner.ts). native-compose-vm-file-client.ts and native-compose-adoption-routing-execution.ts treat a single EPERM as "present", which is safe but was not reviewed here for load behavior.
• The carrier-state call-site cases still select inspections by order. The fixture's daemon cannot see which phase an inspect belongs to, so the cases depend on the documented inspection order, as before.

Round 2 (ce5d368): EPERM case budget

The verifier found that "EPERM is never group absence or settlement" in tests/native-compose-storage-command-record.test.ts now waits the full 15 s settlement budget before refusing (15.7 s locally, 15050 ms in hosted CI) inside a fixed 30 s case budget. Its case budget is now the named GROUP_PROBE_REFUSAL_CASE_BUDGET_MS = 60_000, with the reason in its comment. Both rows pass. Only the test budget changed.

Follow-ups from the verifiers are ticketed: two remaining untagged finalizer refusals, and EPERM handling in the vm-file-client and adoption-routing group probes plus the 30 s sibling budgets in the probe-budget file.

Refs HACK-1463

🤖 Generated with Claude Code

[linear-code]

HACK-1463