Skip to content

Fix CTest helper discovery and isolate unit-test fixtures - #2215

Open
IanHollow wants to merge 1 commit into
getsentry:masterfrom
IanHollow:codex/ctest-isolation-b09d7282
Open

IanHollow wants to merge 1 commit into
getsentry:masterfrom
IanHollow:codex/ctest-isolation-b09d7282

Conversation

@IanHollow

@IanHollow IanHollow commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

CTest's default build does not place the unit executable next to the backend helper it discovers at runtime. Building only sentry_test_unit also does not build that helper. Once the helper is supplied manually, individual CTest cases still race over the same filesystem fixtures under ctest --parallel.

 build sentry_test_unit
+  build the selected helper and place the unit executable alongside it
 run a CTest case
-  reuse the tests/unit working directory
+  create a fresh, nonempty working directory for this invocation
+  remove it on success; retain it and report its path on failure

Preserve CMake's emulator/launcher prefixes and CMP0158 behavior. Add regression coverage for default output paths with spaces, both helper targets, parallel execution, aggregate mode, command prefixes, and repeated success/failure fixture handling. This follows the individual CTest registration introduced in #1244; it changes test setup only.

Evidence

Tested against master bbbd78cd714e282c900d1d19aa75da6b93962e98 (SDK 0.17.4), with initialized recursive submodules.

  • Before: a Crashpad unit-only build fails sentry/basic_function_transport (zero transport calls rather than two). Building/copying the handler manually still leaves six failing cases in a parallel CTest run: cache_max_size, crashed_last_run, installation_id, multiple_inits, old_run_retry, and on_crashed_last_run.
  • After: all 614 Crashpad CTest registrations pass on Linux x86-64 and native macOS arm64, serially, in parallel, and with two consecutive parallel repetitions. Aggregate mode passes on both. All 614 native-backend cases also pass on macOS in those modes.
  • All five added pytest regressions pass (CMAKE_BUILD_PARALLEL_LEVEL=8 CTEST_PARALLEL_LEVEL=8 pytest -q tests/test_ctest.py), including prefix arguments containing spaces and fresh-directory success/failure behavior.
  • make format and make style pass with the pinned formatter requirements. Valgrind reports zero errors for path_basics, path_directory, and path_remove_all_symlink, invoked through the forwarded CTest launcher with full leak checking and definite leaks treated as errors.

Linux native-backend validation exposes an independent existing failure in minidump_structures_packed: the x86-64 context-size assertion at test_native_backend.c:766. This PR does not change the structure or assertion. The same assertion fails in a freshly built, unmodified upstream baseline. Ninja Multi-Config Debug and RelWithDebInfo each pass the other 613/614 registrations; this assertion is the only failure in both. The native regression parameter focuses on helper discovery and shared filesystem cases; none and crashpad exercise the complete CTest set and aggregate mode.

Follow-up: the independent minidump assertion is corrected separately in #2216. Its test arithmetic expected 1244 bytes for the correct 1232-byte AMD64 structure. A disposable combined-source build of both patches passes all 614 Linux native CTest registrations in parallel and with two consecutive repetitions. This PR remains focused on CTest setup.

Windows/cross-target execution and the full HTTP integration suite were not run locally. Native builds used the hosts' configured parallelism; no SDK runtime code or installed library behavior changes.

Merge Danger

Door: two-way. Reverting restores the previous test setup.

Blast Radius: tests. The unit executable's build-tree output follows its selected helper, and CTest cases gain invocation-specific working directories. Failed fixture directories intentionally remain for diagnosis.

@jpnurmi

jpnurmi commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Hi, thanks for the proposal. With a flat RUNTIME_OUTPUT_DIRECTORY (as used by make test-unit and the pytest harness) it should work out of the box, although I agree it'd be nice to make it work out of the box with a hierarchical build tree too.

The project generally uses add_custom_command(POST_BUILD) to copy files in place for tests, so I'd propose using the same approach to deploy the handler next to the unit test executable where appropriate.

Also, could we keep it focused on the minimal CMake fixes needed to run the existing unit tests through CTest? A dedicated integration suite for the CTest wiring is unnecessarily heavy for this scope. 🙂

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants