Skip to content

Handle kqueue cancellation after event fetch - #3518

Closed
HanpuLi wants to merge 5 commits into
python-trio:mainfrom
HanpuLi:fix/kqueue-guest-cancel-stale-event
Closed

HanpuLi wants to merge 5 commits into
python-trio:mainfrom
HanpuLi:fix/kqueue-guest-cancel-stale-event

Conversation

@HanpuLi

@HanpuLi HanpuLi commented Sep 19, 2026

Copy link
Copy Markdown

Fixes #3500.

On kqueue backends, guest mode can fetch an I/O event and then run a host callback before Trio processes that event. If the callback cancels the corresponding wait, the successful abort removes its registration, leaving the already-fetched event with no matching key when process_events() runs.

Track keys removed by successful cancellation. process_events() snapshots and clears that set for the current batch, and ignores a missing registration only when its key was cancelled in that window. Unrelated unknown keys still raise KeyError rather than being silently swallowed.

The regression test reproduces the guest-mode cancellation race on kqueue, with a separate test pinning the unknown-key behavior. CI is green and Codecov reports full coverage of the changed lines.

@codecov

codecov Bot commented Sep 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00000%. Comparing base (50b9825) to head (55753d8).

Additional details and impacted files
@@               Coverage Diff               @@
##                 main        #3518   +/-   ##
===============================================
  Coverage   100.00000%   100.00000%           
===============================================
  Files             128          128           
  Lines           19471        19498   +27     
  Branches         1323         1324    +1     
===============================================
+ Hits            19471        19498   +27     
Files with missing lines Coverage Δ
src/trio/_core/_io_kqueue.py 100.00000% <100.00000%> (ø)
src/trio/_core/_tests/test_guest_mode.py 100.00000% <100.00000%> (ø)
src/trio/_core/_tests/test_io.py 100.00000% <100.00000%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@A5rocks A5rocks left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I was hoping we could add some extra state to ensure that the only events we must skip are these. An initial thing I was thinking about was e.g. add the relevant key to a list when being cancelled, clear at the start of process_events, and ensure anything we skip is in that list.

Does that make sense? Maybe there's something I'm missing, or maybe it takes too many lines to implement for what should never happen...

Comment thread src/trio/_core/_io_kqueue.py Outdated
self._force_wakeup.drain()
continue
receiver = self._registered[key]
receiver = self._registered.get(key)

@A5rocks A5rocks Sep 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is non-obvious, could you cite the test here? e.g. just say # see <test name>. (alternatively, implementing my suggestion would also be enough explanation, I think)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — followed up in 6e9ce05. process_events() now only skips keys that were actually removed by cancellation for that batch, and an unknown-key regression test verifies unrelated missing registrations still raise. The CI run on that head is green.

@HanpuLi

HanpuLi commented Sep 19, 2026

Copy link
Copy Markdown
Author

Thanks, that makes sense. I’ve updated the follow-up so cancellation records only the key whose unregister actually succeeded; process_events() snapshots and clears those keys for that batch, and an unknown key still raises KeyError. I also added a regression test for the unknown-key case.

@HanpuLi

HanpuLi commented Sep 19, 2026

Copy link
Copy Markdown
Author

Follow-up is in: cancellation now records only keys whose abort actually succeeded, and process_events() only skips those keys for that event batch. I also added a regression test confirming unrelated unknown keys still raise KeyError. The latest small follow-up keeps that kqueue-only test out of Linux type checking.

Comment thread src/trio/_core/_tests/test_io.py Outdated
check(expected_monitors=0, expected_readers=1, expected_writers=0)


if sys.platform not in {"win32", "linux"} and not TYPE_CHECKING:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not do pytest.mark.skipif here?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. The earlier skipif version tripped the Linux mypy pass on the kqueue-only names. I restored skipif in 55753d8 and used the existing attr-defined/unused-ignore pattern for those platform-specific attributes. The two focused kqueue tests pass, and mypy is clean for linux, darwin, and win32.

@A5rocks

A5rocks commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Thanks for all the work! I'm going to close this in favor of #3519 since it solves this and more.

@A5rocks A5rocks closed this Sep 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KeyError in KqueueIOManager.process_events (TrioInternalError)

3 participants