Skip to content

fix(env): close the remaining raw env write refusals and publish without hard links - #468

Merged
roodboi merged 1 commit into
nextfrom
wf/g73-env-write-refusals-exfat
Oct 24, 2026
Merged

roodboi merged 1 commit into
nextfrom
wf/g73-env-write-refusals-exfat

Conversation

@roodboi

@roodboi roodboi commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Refs HACK-1377

This PR fixes the env write refusals that were still raw on next after #402 (HACK-1330). All five were pre-existing; none are regressions. Each one now prints a single line with an E_* code and no value, stack, errno or staging path. --json reports the same code.

Root causes and fixes

  1. Filesystems without hard links (ExFAT, FAT). Hack publishes lock receipts (withProjectsRegistryLock) and the first project key (publishPrivateFileExclusive) with link(2). ExFAT returns ENOTSUP (Linux FAT returns EPERM), so the key could never be created. On current next the problem is wider: since fix(env): serialize env layer read-modify-write so concurrent sets keep every key #435, every env set on ExFAT fails at the per-layer lock, including a non-secret one. The output was a raw ENOTSUP … link '<…>.owner' -> '<…>.lock' with a stack.
    Fix: a new src/lib/exclusive-publish.ts. When link reports that hard links are unsupported (ENOTSUP, EOPNOTSUPP or EPERM), it falls back to an exclusive placeholder. Hack creates an empty O_CREAT|O_EXCL|O_NOFOLLOW file at the target, then renames the complete, fsynced sibling over it. The no-overwrite and no-follow guarantees still hold. An existing entry or symlink gets EEXIST. The rename replaces only the empty file this call created, and that is checked again just before the rename. Readers see nothing, an empty file or the complete file, never a partial one. If the rename fails, the reservation is removed. The registry lock now accepts that the rename already used up its staging name. Since the fallback keeps every guarantee, no E_UNSUPPORTED_FILESYSTEM refusal was needed.
  2. Hydrate with a dangling .hack/.internal symlink reached mkdir -p and got a raw EEXIST (E_UNEXPECTED under --json).
    Fix: ensureDirForWrite maps EEXIST to a closed not_a_directory_entry refusal. The hydrate lock now uses the new ensureRealDirectoryForWrite. It refuses a dangling or live symlink, or a file, at .internal as E_PATH_CONFLICT before anything is created through it.
  3. A very long --env name (215 characters) produced lock and staging file names over NAME_MAX, so set, unset and hydrate printed a raw ENAMETOOLONG with the staging path.
    Fix: parseEnvConfigSelection caps overlay names at 128 characters (ENV_CONFIG_NAME_MAX_LENGTH). The longest derived name adds 68 bytes. The CLI refuses longer names up front with the fixed Invalid --env value. message (E_USAGE), and the daemon route uses the same parser. ENAMETOOLONG is also classified as a write failure (name_too_long, E_PATH_CONFLICT) for deep project paths.
  4. A stale key lock. A reused PID is never reclaimed, so ProjectsRegistryLockTimeoutError escaped with a stack.
    Fix: a new ProjectEnvKeyLockTimeoutError with a new code, E_LOCKED. It names the lock and the remedy, and also the .recovery guard when that guard is what is held. ProjectsRegistryLockTimeoutError gains a recoveryBusy field. Its message text is unchanged, because fix(env): name unreclaimable layer locks and the re-include cause #453 (G69) classifies the per-layer lock by that text.
  5. A .hack/.gitignore that is a directory printed a line with no code, and --json gave E_UNEXPECTED.
    Fix: HackDirGitignoreRefusalError is now a HackCliError with E_PATH_CONFLICT, which is what docs/env.md already said. Its wording is unchanged.
  6. Test gaps are now covered: the ENOENT, EPERM and EROFS (and EACCES) entries in isProjectEnvKeyLockUnavailable; the code and message of every AtomicWriteRefusal; behavioural not_regular_file and not_writable refusals; and the staging EISDIR to not_regular_file mapping.

E_LOCKED is added to HACK_ERROR_CODES and the error-code guide, and the plugin reference is regenerated. docs/env.md documents the overlay-name limit, E_LOCKED, the new E_PATH_CONFLICT cases and the behaviour without hard links.

This PR does not touch the per-layer lock message, the unset CAS test, the doctor env-key check or the gitignore-refusal wording. #453 (G69, HACK-1386) owns those. The env-layer lock picks up the fallback and the ENAMETOOLONG mapping only through the shared helpers. git merge-tree of this head with the current #453 head is clean.

Tests

The new tests are in tests/project-env-write-refusal-gaps.test.ts. tests/fixtures/staging-errno-preload.ts gains a link injection step.

  • (1) With link(2) injected as ENOTSUP or EPERM, in and out of git: the first --secret creates a 0600 key, a second set adopts it, the lock is released and nothing is left behind. Concurrent sets of three keys under ENOTSUP keep every key. Unit tests cover the fallback: it publishes and uses up its source, never replaces an existing file, never follows or replaces a symlink, and removes its reservation when the rename fails. A real ExFAT test is opt-in on macOS with HACK_TEST_EXFAT=1, because it attaches a disk image; the image is detached in finally.
  • (2) Hydrate with a dangling or live .internal symlink under all three loggers and --json; unit tests for ensureDirForWrite and ensureRealDirectoryForWrite.
  • (3) The limit boundary (128 accepted, 129 refused); set, unset and hydrate --json with a 215-character name; ENAMETOOLONG injected at layer staging and at the layer lock.
  • (4) A key lock held by a live unrelated PID gives E_LOCKED. Tests also cover recoveryBusy propagation and both messages.
  • (5) A .hack/.gitignore directory under all three loggers and hydrate --json.
  • (6) As listed above.

Mutation proof: each mutation below was applied alone, its focused tests were run, and the source was restored. All 21 were killed.

Mutation Result
M1 no fallback on unsupported link / M1b drop EPERM from the fallback codes 5 fail / 3 fail
M2 lock unlinks the consumed candidate / M3 failed rename keeps its reservation / M3b placeholder path replaces an existing file 2 / 1 / 2 fail
M4 ensureDirForWrite raw EEXIST / M5 hydrate lock uses plain ensureDirForWrite 1 / 1 fail
M6 no overlay-name limit / M6b off by one (<) / M7 ENAMETOOLONG unclassified 2 / 2 / 1 fail
M8 key lock timeout left raw / M9 recoveryBusy dropped 1 / 1 fail
M10 .hack/.gitignore refusal back to E_UNEXPECTED 1 fail
M11a/b/c drop ENOENT / EPERM / EROFS from isProjectEnvKeyLockUnavailable 1 fail each
M12a–d wrong code for not_regular_file, not_writable, owner_not_preserved, group_not_preserved 2 / 2 / 1 / 1 fail
M13 staging EISDIR mapped to not_a_directory 1 fail

Evidence

  • Real ExFAT, macOS (hdiutil create -fs ExFAT, detached in a trap; private HOME, HACK_HOME and DOCKER_CONFIG; keychain fallback disabled; layer file written first):
    • On origin/next (7d9c89f): raw ENOTSUP … link '…hack.env.default.yaml.lock.<uuid>.owner' plus a stack, exit 1, in a plain directory, a git checkout and a linked worktree.
    • At this head: every case exits 0. A plain directory gets a checkout-local key and env get round-trips the value. A git checkout runs a second --secret set and a non-secret set. In a linked worktree, the key is written to the shared location in .git. No .tmp, .owner or .lock file is left behind.
    • Six concurrent first --secret sets on ExFAT, in a plain directory and in a git checkout, all exit 0, and all six values round-trip.
    • The opt-in HACK_TEST_EXFAT=1 test passes (85 expects).
  • Items (2) to (5), before and after, with the built dist/hack:
    • (2) Before: raw EEXIST … mkdir with a stack, and E_UNEXPECTED. After: E_PATH_CONFLICT: Could not create directory <…>/.hack/.internal: that path is a symlink or not a directory. Replace it with a regular directory, then retry.
    • (3) Before: raw ENAMETOOLONG … open '<…>.owner'. After: the fixed Invalid --env value. line; --json gives E_USAGE. A 128-character name still writes its layer.
    • (4) Before: Timed out waiting for project env key lock with a stack. After: E_LOCKED: … remove <…>/.git/.hack.secret.key.lock, then retry. No key is created, and the retry succeeds once the lock is removed.
    • (5) Before: no code, and E_UNEXPECTED. After: E_PATH_CONFLICT: Refused .hack/.gitignore access: ….
  • Focused tests:
    • project-env-write-refusal-gaps: 28 pass, 1 skip (the opt-in ExFAT test).
    • project-env-write-refusals: 74/74 with --timeout 120000.
    • Nine related files with --timeout 120000 (projects-registry-lock, projects-registry-identity, project-env-layer-lock, project-env-key-ignore, project-env-worktree-key, private-env-file, env-selection-parity, env-key-argument-redaction-cli, focused-skills): 113/113.
  • Gates:
    • bun run check (includes privacy:check): exit 0.
    • bun run --cwd packages/cli typecheck: 0 errors.
    • Biome on the changed tests: clean.
    • gitleaks 8.30.1: gitleaks dir on the git log -p origin/next..HEAD export finds no leaks.
    • bun run generate:agent-plugins and bun run docs:cli-reference produce no further diff.
  • Full suite: the hosted CI test job at this PR head.

Gaps

  • Interrupted fallback publish. Without hard links, a process killed between the reservation and the rename leaves an empty .hack.secret.key or lock file. Hack refuses it and never adopts or reclaims it, so it must be removed by hand. docs/env.md says so. ExFAT and FAT also do not store Unix permissions, so mode 0600 does not limit the key there; this is documented as well.
  • ExFAT test is macOS-only and opt-in. Hosted CI covers the fallback only through the injected ENOTSUP and EPERM cases. Linux FAT (EPERM) is covered only by injection.
  • Help text after a usage error. A too-long --env prints the same one-line ERROR on stderr as any other invalid --env, plus the existing usage help on stdout. That is the pre-existing usage-error convention, and this PR leaves it unchanged.
  • Local test runs were affected by host load. This host ran at a load average of 50 to 290. Default-timeout runs of project-env-worktree-key, project-env-key-ignore and project-env-write-refusals failed with 5-second hook timeouts. The same project-env-worktree-key file also fails on an untouched origin/next export under the same load (8 of 14). With a 120-second timeout, all of these files pass. Hosted CI is the full-suite evidence.
  • New error code. E_LOCKED is new. fix(env): name unreclaimable layer locks and the re-include cause #453 leaves the per-layer lock refusal without a code. If it wants one, it can reuse E_LOCKED once both PRs land.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

…out hard links

On a filesystem without hard links (ExFAT, FAT) every env write failed
with a raw ENOTSUP from the lock receipt's and the first project key's
link(2) publish, so the key could never be created there. The exclusive
publish now falls back to an O_CREAT|O_EXCL|O_NOFOLLOW empty reservation
that the complete, fsynced sibling is renamed over, keeping the
no-overwrite and no-follow guarantees.

The other refusals that were still raw on next are now one value-free
line with an E_* code and no stack:

- hydrate with a symlinked (dangling or live) or non-directory
  .hack/.internal is E_PATH_CONFLICT; nothing is created through it
- an --env overlay name over 128 characters is refused up front as
  E_USAGE, and ENAMETOOLONG is classified as E_PATH_CONFLICT
- a project env key lock that stays held is E_LOCKED, naming the lock
  (and its recovery guard when that is held) and the remedy
- a .hack/.gitignore directory is E_PATH_CONFLICT, as docs/env.md said

Adds mutation-proven tests for these and for the previously untested
lock-unavailable codes, atomic-write refusal codes and the staging
EISDIR mapping.

Refs HACK-1377

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Oct 24, 2026

Copy link
Copy Markdown

HACK-1377

@roodboi
roodboi marked this pull request as ready for review October 24, 2026 21:07
@roodboi
roodboi merged commit b54f38d into next Oct 24, 2026
17 checks passed
@roodboi
roodboi deleted the wf/g73-env-write-refusals-exfat branch October 24, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant