Skip to content

fork-ci: green baseline on fork/main - #19

Merged
thedancingdeveloper merged 3 commits into
fork/mainfrom
fork/ci-baseline-2
Oct 10, 2026
Merged

thedancingdeveloper merged 3 commits into
fork/mainfrom
fork/ci-baseline-2

Conversation

@thedancingdeveloper

@thedancingdeveloper thedancingdeveloper commented Oct 10, 2026 •

Copy link
Copy Markdown

Makes fork-ci pass on code already in fork/main, so feature PRs (#6, #10, #12, #13, #14) stop failing on tests they don't touch. Tracked as Vogt WI-1157.

Why each PR showed a different single failure

nextest is fail-fast by default, so fork-ci cancelled a crate's remaining tests at its first failure (3620/5075 tests run … 1 failed). Each PR surfaced one unrelated failure, and fixing it only exposed the next. The first commit adds --no-fail-fast, which also makes good on the step's own comment that "a PR should see all of its failures at once".

Failures on unmodified fork/main (af8bd29) and how each is handled

Failure Cause Handling
codex-tui: 36 tests in CI (37 locally), including app::daemon_menu::tests::daemon_menu_is_read_only_and_confirmation_can_cancel_or_handoff, status::tests::status_snapshot_*, history_cell::tests::*update_available*, update_prompt::tests::* Stale expectation from the base. fork/main sits on upstream tag rust-v0.162.0. Its release commit c138238 bumps the workspace version from 0.0.0 to 0.162.0, and Cargo.lock keeps 0.0.0. Upstream tests at 0.0.0 and never on the tag commit, so the snapshots render v0.0.0. Every diff is only 0.0.0 → 0.162.0. fork-ci resets the workspace version to 0.0.0 for the job, the same state upstream tests in. This edits only fork-ci.yml, so it survives every rebase; re-snapshotting would conflict on each new tag. fork-release stamps the real version separately. Locally all 5710 codex-tui tests pass at 0.0.0; 37 fail at 0.162.0.
codex-cli: app_server_daemon::packaged_daemon_bootstrap_seeds_local_package (autoUpdateEnabled true, expected false) Same base-version cause. prepare_install::stable_version treats only 0.0.0 as a dev build. At 0.162.0 the seeded test package counts as a stable release and follows latest, so the updater is enabled. Fixed by the same 0.0.0 reset.
codex-core: suite::hooks::pre_tool_use_merges_hooks_json_and_config_toml Stale expectation from merged fork change #5 (3e2cb85), which added the resolved workdir to exec_command's PreToolUse tool_input. This test compares the whole tool_input and expected only command. The assertion now expects workdir equal to the session cwd, because the call sets no workdir. Passes locally.
codex-core: suite::hooks::permission_request_hook_* Flaky. Each passed on retry (FLAKY 2/2) in earlier fork-ci runs, under nextest's retries = 1. No change; the retry already absorbs it.
codex-core: all__suite__scenarios__astra_* snapshots Not reproduced. All 9 astra scenario tests pass on unmodified fork/main locally, and no astra .snap.new is left in any fork worktree. No change. The CI baseline run also has no astra failure.

.github/fork-ci-known-failures.txt stays empty: nothing needed excluding.

Evidence

  • Local reproduction on unmodified af8bd29 with fork-ci's exact steps (bins built first, the same rusty_v8 artifacts, RUST_MIN_STACK, nextest 0.9.103, --no-fail-fast). Local-only failures came from the dev host, not the code: its AppArmor blocks bwrap namespaces, which CI enables, and the daemon app-server doesn't start there. CI is authoritative for those.
  • CI baseline on the unmodified code with --no-fail-fast, run 38016341231: exactly 38 failures, namely codex-tui 36 (5710 run), codex-cli 1 (packaged_daemon_bootstrap_seeds_local_package, 493 run) and codex-core 1 (pre_tool_use_merges_hooks_json_and_config_toml, 5075 run, 1 flaky that passed on retry). Every one is in the table above.
  • CI on this branch for codex-tui, codex-cli and codex-core: run 38017947571.

Stray change: any cargo command rewrites Cargo.lock from 0.0.0 to 0.162.0 at this base. That rewrite is not committed, and with the reset it no longer happens in CI.

🤖 Generated with Claude Code

thedancingdeveloper and others added 3 commits October 10, 2026 02:16
nextest is fail-fast by default, so fork-ci cancelled a crate's remaining
tests at its first failure. Each PR run therefore surfaced a single
unrelated failure per crate, and fixing it only exposed the next one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fork/main sits on upstream tag rust-v0.162.0, whose release commit
(c138238) bumps the workspace version from 0.0.0 to 0.162.0 and leaves
Cargo.lock at 0.0.0. Upstream runs its tests at 0.0.0 and never on that
commit, so on fork/main 37 codex-tui snapshots rendered v0.162.0 instead
of v0.0.0, and packaged_daemon_bootstrap_seeds_local_package saw the
seeded package as a stable release and enabled auto-update.

Reset the workspace version to 0.0.0 for the job, the way upstream tests.
fork-release stamps the real version separately. All 5710 codex-tui tests
pass at 0.0.0 locally; 37 fail at 0.162.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3e2cb85 added the resolved workdir to exec_command's PreToolUse
tool_input, but pre_tool_use_merges_hooks_json_and_config_toml compares the
whole tool_input and still expected only command. The call sets no
workdir, so it resolves to the session cwd.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thedancingdeveloper

Copy link
Copy Markdown
Author

Oversight review: no blockers (head 92886e1)

  • The workflow resets the workspace version to 0.0.0 only inside the fork-ci job, with a guard that fails if the reset doesn't apply. fork-release is untouched. This matches how upstream tests, and it survives rebases onto new tags.
  • --no-fail-fast per crate means PRs see every failure at once.
  • The hooks assertion now expects workdir = session cwd, matching merged fork change Include the resolved workdir in the exec_command PreToolUse payload #5.
  • fork-ci is green on this head.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

@thedancingdeveloper
thedancingdeveloper merged commit c233dbb into fork/main Oct 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant