diff --git a/.github/workflows/fork-ci.yml b/.github/workflows/fork-ci.yml index 1b8e0a0e1601..f21fe8921a0b 100644 --- a/.github/workflows/fork-ci.yml +++ b/.github/workflows/fork-ci.yml @@ -6,6 +6,10 @@ name: fork-ci on: pull_request: branches: [fork/main] + # The merge queue (WI-1886, docs/ci.md). The queue tests the tree fork/main + # will become, so a PR never has to be rebased onto a sibling's merge. + merge_group: + types: [checks_requested] workflow_dispatch: inputs: # Comma-separated package names to check instead of the diff. Used to @@ -18,6 +22,12 @@ on: permissions: contents: read +# A force-push cancels the superseded run. Each queue entry has its own +# gh-readonly-queue/... ref, so queue entries never cancel each other. +concurrency: + group: fork-ci-${{ github.ref }} + cancel-in-progress: true + jobs: check: name: fmt, clippy, test @@ -106,11 +116,27 @@ jobs: id: crates env: FORCED_PACKAGES: ${{ inputs.packages }} + QUEUE_BASE: ${{ github.event.merge_group.base_sha }} shell: bash run: | set -euo pipefail - base="$(git merge-base HEAD origin/fork/main 2>/dev/null || git merge-base HEAD fork/main)" + # In the queue, diff the whole group against the queue's base: every + # PR ahead of this one in the group is part of the tree under test. + if [[ -n "$QUEUE_BASE" ]]; then + base="$QUEUE_BASE" + else + base="$(git merge-base HEAD origin/fork/main 2>/dev/null || git merge-base HEAD fork/main)" + fi mapfile -t files < <(git diff --name-only "$base"...HEAD) + # Path scope: a diff with nothing under codex-rs/ or .github/ (docs, + # FORK.md, sdk/) skips the Rust steps. The job still reports, so a + # required check is satisfied. + rust=false + for f in "${files[@]}"; do + case "$f" in codex-rs/*|.github/*) rust=true ;; esac + done + [[ -n "$FORCED_PACKAGES" ]] && rust=true + echo "rust=$rust" >> "$GITHUB_OUTPUT" if [[ -n "$FORCED_PACKAGES" ]]; then mapfile -t packages < <(echo "$FORCED_PACKAGES" | tr ',' '\n') echo "forced: ${packages[*]}" @@ -194,6 +220,7 @@ jobs: # the Codex-built artifacts from the upstream release and verifies them # against the in-tree checksum manifest, the same as fork-release. - name: Configure rusty_v8 artifact overrides and verify checksums + if: steps.crates.outputs.rust == 'true' uses: ./.github/actions/setup-rusty-v8 with: target: x86_64-unknown-linux-gnu @@ -203,9 +230,11 @@ jobs: # apt package satisfies it. Upstream never builds it with cargo either; it # builds //codex-rs/voice-host with Bazel against a bundled GStreamer. - name: cargo check + if: steps.crates.outputs.rust == 'true' run: cargo check --workspace --all-targets --exclude codex-voice-host - name: cargo fmt + if: steps.crates.outputs.rust == 'true' shell: bash run: | set -euo pipefail @@ -221,10 +250,13 @@ jobs: # PR changed fails the build. Compile errors are fatal regardless, which # `cargo check` above already enforces. - name: cargo clippy + if: steps.crates.outputs.rust == 'true' shell: bash + env: + QUEUE_BASE: ${{ github.event.merge_group.base_sha }} run: | set -euo pipefail - base="$(git merge-base HEAD origin/fork/main 2>/dev/null || git merge-base HEAD fork/main)" + base="${QUEUE_BASE:-$(git merge-base HEAD origin/fork/main 2>/dev/null || git merge-base HEAD fork/main)}" # One changed line range per hunk, as "file start count" with the # codex-rs/ prefix stripped. A count of 0 is a pure deletion. git diff -U0 "$base"...HEAD -- codex-rs \ @@ -274,6 +306,7 @@ jobs: fi - name: cargo test + if: steps.crates.outputs.rust == 'true' shell: bash # codex-core's tests overflow the default thread stack (fatal runtime # error: stack overflow). Upstream sets the same 8 MiB floor on its diff --git a/FORK.md b/FORK.md index ef9e553a3af2..e3b61347b4b8 100644 --- a/FORK.md +++ b/FORK.md @@ -94,3 +94,8 @@ runs GitHub-hosted. `fork-upstream-drift` is the check that the recorded base ta "Upstream fixes taken through" marker, is still the newest stable tag (WI-1125). It has no schedule: run it on `workflow_dispatch`, or from the sync runbook. + +`fork-ci` supports a merge queue (`merge_group`), skips its Rust steps when a +PR touches nothing under `codex-rs/` or `.github/`, and cancels superseded runs +per ref. How it runs on PRs, in the queue, and around upstream syncs: +`docs/ci.md`. diff --git a/docs/ci.md b/docs/ci.md new file mode 100644 index 000000000000..c8f352e36244 --- /dev/null +++ b/docs/ci.md @@ -0,0 +1,61 @@ +# Fork CI: pull requests, the merge queue, and `fork/main` + +Fork-only. Upstream's CI is disabled on this fork (see `FORK.md`, +"Workflows"); `fork-ci` is the one check that runs on pull requests into +`fork/main`. This file covers how it is meant to run with a merge queue +(WI-1886). + +## What runs where + +| Event | `fork-ci` | +|---|---| +| `pull_request` into `fork/main` | Rust steps only when the diff touches `codex-rs/` or `.github/`; fmt/clippy/nextest only for the packages the diff changes | +| `merge_group` (the queue) | Rust steps for the packages the **whole queued group** changes, diffed against the queue's base (`merge_group.base_sha`), plus `cargo check --workspace` | +| `workflow_dispatch` | the packages named in `packages`, or the diff | + +A pull request that touches only docs, `FORK.md` or `sdk/` still reports +`fmt, clippy, test` as passed, with the Rust steps skipped, so a required +check never waits on it. + +### Why the queue is package-scoped too + +Vogt's queue runs everything. Here, "everything" is a workspace-wide nextest +over hundreds of upstream crates, at roughly an hour on the 30-vCPU runner, and +upstream lint and test debt makes a full run fail regardless of the PR. The +queue keeps the package selection but widens the diff to the whole group, and +`cargo check --workspace --all-targets` still compiles every crate, so a +cross-crate API break fails the queue. + +### Concurrency + +`fork-ci` cancels in progress per ref. A force-push to a PR cancels the +superseded run. Each queue entry has its own `gh-readonly-queue/fork/main/...` +ref, so entries never cancel each other. + +### Runners + +Same-repo PRs, queue entries and `workflow_dispatch` run on the self-hosted GCP +Spot `rust-heavy` runner. A PR from another fork stays on GitHub-hosted +`ubuntu-24.04`, so untrusted code never reaches a persistent runner. A +`merge_group` event is always trusted: only a maintainer can enqueue. + +## Landing + +`land-pr` (workspace `.agent-tools/bin/land-pr`) detects a merge queue from +the base branch's rulesets. It enqueues with `gh pr merge --auto` and follows +the entry's `merge_group` run. Don't rebase a queued PR by hand. + +## Upstream syncs + +A rebase of `fork/main` onto a new upstream tag is a force-push +(`docs/fork-upstream-sync.md`), not a queued merge. Drain or pause the queue +before you push the rebase. The ruleset below bypasses only the repository +admin role, which is what lets the `--force-with-lease` push through. + +## Turning the queue on (operator) + +`fork/main` has no branch protection and no ruleset today, so `fork-ci` is not +required and the queue is off. The workflow supports the queue without +needing it. The exact ruleset is in the WI-1886 PR description: one branch +ruleset on `fork/main` that requires `fmt, clippy, test` and adds a +`merge_queue` rule.