New sessions can't be created: git fetch during worktree init deadlocks on the fsmonitor daemon → "git command timed out after 60 seconds"

[brian-thomas-hpe]

Short summary

New sessions can't be created: git fetch during worktree init deadlocks on the fsmonitor daemon → "git command timed out after 60 seconds"

Affected version or release

1.1.23 (regression introduced in 1.1.12; also reproduced on 1.1.16 and 1.1.22)

Installation context

macOS desktop app, single-user local install. Affects a normal clone that has core.fsmonitor=true and feature.manyFiles set. Copilot CLI 1.0.87-0 (bundled with app 1.1.23), Bundled git: 2.53.0-4 (the -4 build; system git is 2.98.0), Mac: M3 Pro macOS Sequoia 15.7.7

What happened?

Since v1.1.12, creating a new session fails with:

Session '' workspace initialization failed: git command timed out after 60 seconds

The worktree is never created, so no session is produced. It persists across app restarts and across the two updates I applied today (1.1.16 → 1.1.22 → 1.1.23). Deleting ~12 stale sessions did not help.

Root cause: during init the app runs (env-injected; no -c core.fsmonitor= in the argv):

git -C fetch --progress --no-write-fetch-head origin +refs/heads/:refs/remotes/origin/

With core.fsmonitor effectively on, the fetch spawns/attaches a git fsmonitor--daemon that inherits the fetch's --progress stderr write-end and never closes it. The parent holds the read-end waiting for progress; because the daemon keeps the write-end open, the pipe never reaches EOF. The watchdog sees received_output=false and kills at idle_timeout_secs=60.

This looks like the same defect as #3980 (empty core.fsmonitor override treated as "missing" at a serialization boundary, so fsmonitor stays on) surfacing on the session-creation path. My captured fetch argv carries no core.fsmonitor override at all.

Steps to reproduce

1. Use a clone with core.fsmonitor=true and feature.manyFiles set.
2. Create a new session (worktree-backed) in that project.
3. Observe init hang ~60s, then "git command timed out after 60 seconds"; no session created.

Reproduced outside the app with a standalone harness that mimics the fetch (parent holds the stderr pipe read-end while running git fetch --progress):

• core.fsmonitor=false → pipe hits EOF in ~2s, fetch exits cleanly.
• core.fsmonitor=true → no EOF, fetch stays alive indefinitely = deadlock.
  Reproduces identically on both bundled git builds (2.53.0 -3 and -4), so the bundled git version is not the cause.

Expected behavior

Session creation completes: the init fetch finishes and the worktree is created, regardless of whether the clone has core.fsmonitor enabled.

Additional context

Version/onset from ~/.copilot/logs (per-launch version= strings correlated with github_app::git::progress watchdog events):

• ≤ v1.1.11: 0 idle-timeout events (clean)
• v1.1.12 (installed Sep 9): 16 events; first fire 2026-09-16T18:16:31Z; idle_timeout_secs changed 300 → 60; hits on the refresh/worktree_sync path
• v1.1.16: 20 events; first create-path hangs 2026-09-21T14:35Z via create_worktree_workspace / workspace_kickoff_after_init
• v1.1.22 → v1.1.23 (today): ongoing
  The fetch command string is byte-identical back to July, so the trigger is the surrounding process/env handling + the 300→60s watchdog drop, not the git invocation.

Log signatures: github_app::git::progress ("idle timeout — no stderr activity", idle_timeout_secs=60, received_output=false); create path create_worktree_workspace / workspace_kickoff_after_init; refresh path worktree_sync. The app does not log child-process env, so the GIT_CONFIG_COUNT / core.fsmonitor injection is invisible in logs — please confirm whether the empty override actually reaches the child (the #3980 serialization concern).

Related issues:

• GIT_CONFIG_COUNT is driving me f'ing crazy it keeps throwing errors breakin... #3980 — almost certainly the same root defect (empty core.fsmonitor override dropped as "missing"); this is that defect on the session-creation path.
• Error toast after deleting sessions. - Failed to delete session: git ["fsmonitor--daemon", "stop"] failed: fatal: fsmonitor--daemon is not running #3156 — "git ['fsmonitor--daemon','stop'] failed: fatal: fsmonitor--daemon is not running"; sibling symptom proving buggy per-worktree fsmonitor lifecycle management.
• Background git fetch every ~5s per worktree: high process churn with many worktrees, and the local branch is never updated #2403 — documents the exact init fetch command and ~5s/worktree fetch cadence; reporter left fsmonitor off (plausibly why the empty-override default was adopted — now broken per GIT_CONFIG_COUNT is driving me f'ing crazy it keeps throwing errors breakin... #3980).

Suggested fixes:

1. Ensure the fsmonitor override reaches the child — pass -c core.fsmonitor=false in the argv, not only via an env override an empty-string-as-missing boundary can drop.
2. Don't let the spawned fsmonitor daemon inherit the fetch's stderr write-end — redirect the daemon's stderr or close the write-end in the parent after spawn so the progress pipe reaches EOF.
3. Make the watchdog treat process exit (not only stderr activity) as completion; the 300→60s drop in v1.1.12 just made an existing pipe hazard fail faster.

Workaround: setting core.fsmonitor false (and feature.manyFiles false) on the affected clone stops the daemon from spawning and lets init fetch complete. Per-clone (--local); does not affect other clones of the same repo.

[brian-thomas-hpe]

Update — precise root cause confirmed (refines the mechanism in the original report)

Follow-up on this issue with tighter instrumentation. I can now pin the exact failure path, and it's more specific — and reliably reproducible — than the mechanism I proposed above. The user-facing "workspace initialization failed: git command timed out" is a local git read-tree -mu HEAD checkout stalling on an app-forced fsmonitor daemon, not a network/--progress fetch.

Definitive evidence

1. The app writes core.fsmonitor=true onto the managed clone at worktree-create time.

   • I set git config --local core.fsmonitor false on the clone at 20:40:28Z and verified it held.
   • On the next session create, .git/config was rewritten and core.fsmonitor was true again — with an mtime of 20:41:41Z, matching the worktree_create ... deferred_checkout=true log line to the second. (untrackedCache/feature.manyFiles are also set to true.)

2. The deferred checkout then hangs on the fsmonitor daemon.

   • Task worktree_deferred_checkout runs git_command{command="read-tree" arg_count=3 timeout_secs=300} → ["read-tree", "-mu", "HEAD"].
   • It spawns a fresh git fsmonitor--daemon run --detach --ipc-threads=8 and blocks on its IPC. Heartbeats log git command still running ... elapsed_ms=30002/60001/90002 with 0 files materialized in the new worktree.

3. Killing the daemon makes read-tree fall back and complete immediately.

   • kill <daemon pid> → read-tree exits, all files materialize, and the log emits deferred worktree checkout complete ... populate_ms=91817. Worktree ends clean on the correct branch.
   • This isolates the fsmonitor daemon as the sole blocker.

4. Out-of-app repro: a plain git -c core.fsmonitor=true status/checkout against the same clone hangs identically with bundled git 2.53.0-4; with core.fsmonitor=false it's instant.

Why restarting the app never helps

The fsmonitor--daemon processes are setsid-detached, so they survive app quit. They accumulate (I observed 13 live daemons, the oldest 45 days), plus a git fsmonitor--daemon stop wedged for 2h+. Every restart inherits the same broken daemon layer. This directly corroborates #3156 (fsmonitor--daemon stop failed: fatal: fsmonitor--daemon is not running) and the broad core.fsmonitor propagation described in #3980. It's also consistent with the per-worktree fsmonitor cost documented in #2403.

Environment

• App 1.1.23 (commit d1a29ed), macOS (aarch64)
• Copilot CLI 1.0.87-0
• Bundled git 2.53.0-4 (also a stale 2.53.0-3 daemon still alive); system git 2.98.0
• Onset tracks the v1.1.x line where core.fsmonitor began being applied broadly and the git-command idle timeout dropped 300s→60s; create-path checkouts began hanging shortly after.

Suggested fixes (any one resolves it)

1. Don't force core.fsmonitor=true on managed clones — respect the user's config, or make it opt-in.
2. Run the deferred checkout with -c core.fsmonitor=false so read-tree/checkout never depends on the daemon.
3. Terminate spawned fsmonitor--daemon processes on app exit (fixes the survive-quit pileup / Error toast after deleting sessions. - Failed to delete session: git ["fsmonitor--daemon", "stop"] failed: fatal: fsmonitor--daemon is not running #3156).
4. Bounded fallback: if read-tree stalls, retry once with fsmonitor disabled instead of running to the 300s hard timeout.

Workaround for affected users

When a session's checkout hangs, kill the git fsmonitor--daemon process for the new worktree (explicit PID). The blocked read-tree immediately falls back and the checkout completes. Setting core.fsmonitor false on the clone alone is not sufficient, because the app rewrites it to true on the next worktree create.