Skip to content

fix: drain request disconnect tasks on cancellation - #528

Open
cloudgpuapp wants to merge 1 commit into
theroyallab:mainfrom
cloudgpuapp:fix/drain-disconnect-tasks-r22
Open

cloudgpuapp wants to merge 1 commit into
theroyallab:mainfrom
cloudgpuapp:fix/drain-disconnect-tasks-r22

Conversation

@cloudgpuapp

Copy link
Copy Markdown

Is your pull request related to a problem? Please describe.

run_with_request_disconnect() waits for the supplied task or a disconnect watcher. If its own caller is cancelled during that wait, it exits without cancelling either child. Its normal/error/disconnect paths also request cancellation without awaiting child cleanup.

The helper is used by embeddings, embedding-model loading and model downloads. This change does not modify the streaming chat/completions path or claim to fix #502.

Why should this feature be added?

Keep explicit references to both tasks and cancel/drain them in finally, including when the outer request task is cancelled. Preserve the existing result, call exception and HTTP 422 behavior. Outer task cancellation continues to propagate as CancelledError.

Examples

The regression test starts both tasks behind Events, cancels the wrapper and checks that both children have finished their asynchronous cleanup before it exits. Another case holds a child's cleanup behind an Event, verifies the wrapper is still pending, then releases it in finally so no test task is left behind.

Additional context

  • Python 3.12.16, macOS arm64; ordinary imports of the real common.networking and its configuration/logger dependencies. The work and disconnect coroutines are controlled test doubles; no AST extraction, native backend stub, model, network download or GPU workload.
  • Original implementation: all six new lifecycle tests fail. Patched implementation: all six pass.
  • Ruff 0.11.10: full repository lint and format checks pass (134 files).
  • Expanded checks with existing port tests: 11 passed / 1 failed. The same wildcard-bind assertion also fails on the unmodified baseline on this Mac (5 passed / 1 failed in the existing port suite). The patch does not change port_bind_error. Full model/server tests were not run.
  • Cancellation remains cooperative. Waiting for cleanup has no fixed time bound if a child suppresses cancellation or blocks. It cannot force-stop native kernels, threads, or a task cancelled before this wrapper ever starts. Repeated cancellation during cleanup was not tested.

Prepared with AI assistance; no provider integration or promotional links are added.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[NO BUG] chat/completions (stream) is impossible to cancel

1 participant