Skip to content

toolchain_includes: verify include roots a compiler wrapper/sysroot injects - #25

Open
bluecmd wants to merge 3 commits into
EngFlow:mainfrom
bluecmd:up-toolchain-includes
Open

bluecmd wants to merge 3 commits into
EngFlow:mainfrom
bluecmd:up-toolchain-includes

Conversation

@bluecmd

@bluecmd bluecmd commented Sep 27, 2026

Copy link
Copy Markdown

AI-generated: the code, tests and this description were written by Claude (Anthropic) agents and reviewed by a Claude session. A human (@bluecmd) directed the work and decided to send it. We hit this migrating CMake packages built with a wrapped cross toolchain (OpenWrt's gcc driver).

Problem. Compiler wrappers, vendor SDK shims and sysroots add include roots when the compiler runs, so bazel aquery never shows them. The reference build often passes the same roots explicitly (e.g. CMAKE_C_FLAGS=-I<sdk>/usr/include), so every TU reports an includes_diff. The only lever today is ignore.include_prefixes, which deletes the root from both sides. That hides the reference losing the root, and stops checking every root beneath it. In our case a prefix ignore on the toolchain dir hid that <toolchain>/include/fortify was missing on the Bazel side.

Change.

  • "toolchain_includes": ["/opt/sdk/usr/include"] (top level of any2bazel.json) is an assertion, not a suppression. Each root is required on every CMake TU, and a missing one is an includes_diff error. It counts as present on the Bazel side. Roots match by exact directory, not prefix, so <sdk>/usr/include/json-c is still checked.
  • scripts/probe_toolchain.py [--env K=V] -- <compiler> runs -v -E and -### on the same compiler and environment the Bazel toolchain uses, without compiling anything. It prints the search lists and injected flags, plus a snippet to paste, so the entries come from the driver rather than from memory.
  • triage.py prints each finding's detail lines, and SKILL.md gains 12 lines: a config-reference entry and the includes_diff fix.

Testing. Tests are in test_engine.py, test_triage.py and the new test_probe_toolchain.py, whose fixtures are captured gcc, clang and cross-driver output. All existing tests and the copyright-header check pass. We also ran the probe against host gcc and against a real mips64 cross toolchain. Projects that don't set the key behave exactly as before.

🤖 Generated with Claude Code

bluecmd and others added 3 commits September 27, 2026 08:09
Toolchains that wrap the compiler add include roots at execution time: a
cross SDK's gcc wrapper adds `-idirafter <sdk>/usr/include` when it runs,
the sysroot's builtin dirs are always searched, and ccache/distcc/vendor-SDK
shims do the same kind of thing. The reference build often spells those
roots out on every TU (CMAKE_C_FLAGS), `bazel aquery` can never show them,
and until now the only lever was ignore.include_prefixes, which deletes the
root from BOTH sides -- so the reference silently losing the root is never
noticed, and every root beneath the prefix (a <sdk>/usr/include/json-c
dependency root) stops being verified as well.

`toolchain_includes` is the asymmetric lever: each declared root is
REQUIRED on every CMake TU (its absence is an includes_diff error -- the
assertion is stale or the reference regressed) and counts as present on
the Bazel side. It matches by directory identity, not by prefix, on
purpose: a subdirectory of a toolchain root is a different search directory
with its own meaning and is compared on its own merits (include_map it if
Bazel spells it differently). A prefix match would swallow it exactly the
way include_prefixes does; on a real cross-compiled project a prefix on the
toolchain dir hid that <toolchain>/include/fortify (where the reference
resolves <string.h> to a fortify wrapper) was absent on the Bazel side. The
toolchain root is decided first in _norm_includes, so neither include_map
nor include_prefixes can rewrite or delete it out of the required check;
both still govern every other root.

triage.py now prints each kind's distinct `detail` lines, because the new
finding (root under bazel_only, severity error) would otherwise sit under
the "usually tolerated" label with nothing saying how to read it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
toolchain_includes entries must be facts about the toolchain, not recall.
The probe runs the very compiler command the Bazel toolchain runs, with the
env it runs under, and reports what the driver actually does:
`-v -E -x <lang> /dev/null` for the effective "..." / <...> search lists,
the cc1 command with every include flag the driver injected and the
driver-level options a wrapper prepended; `-### ... -o /dev/null` for the
-L / -rpath-link / --sysroot the link command gets. Nothing is compiled or
linked. It ends with a toolchain_includes snippet to paste (`--json` for the
raw report). Running it with and without a wrapper's gating environment
variable shows exactly what that variable adds.

Checked against the host gcc and a wrapped cross gcc whose driver adds
`-idirafter <sdk>/usr/include` only when its environment variable is set.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A config-reference entry for the new field (what it asserts on each side,
directory identity rather than prefix, and that the roots come from the
probe, not from memory), a second reading of includes_diff in the fix
table, and the probe's tests in the test command.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant