Skip to content

Do not stop the zstream loop for an interrupt - #151

Open
bruno-costanzo wants to merge 2 commits into
ruby:masterfrom
bruno-costanzo:fix-deflate-finish-after-interrupt
Open

bruno-costanzo wants to merge 2 commits into
ruby:masterfrom
bruno-costanzo:fix-deflate-finish-after-interrupt

Conversation

@bruno-costanzo

@bruno-costanzo bruno-costanzo commented Oct 12, 2026 •

Copy link
Copy Markdown

#121 fixed the Zlib::BufError from #57, but its retry condition lets Zlib::Deflate#finish return before the stream ends. This is the iteration @ioquatix mentioned in #74: the same idea as #74 (@ianks), taken one step further.

Problem

In zlib 3.2.4 (and in Ruby 3.4.9 and later, which ship #121 as zlib "3.2.3"), when a thread is interrupted during a deflate, the output can be truncated:

  • Zlib.deflate returns truncated output and raises nothing.
  • Zlib::Deflate#finish returns truncated output. A later close raises Zlib::DataError, because deflateEnd sees an unfinished stream.
  • Zlib::GzipWriter#close writes truncated output, and then usually raises Zlib::DataError.

An interrupt is Thread#wakeup, or a signal such as SIGCHLD from a backtick in another thread. This partly brings back Bug #10961 ("Zlib corrupts data when receive signal"), which #22 fixed with the retry.

Reproduction (ruby 4.0.6, arm64-darwin, libz 1.3.1):

require "zlib"
require "securerandom"

content = SecureRandom.random_bytes(256 * 1024)
outputs = []

thr = Thread.new do
  100.times do
    z = Zlib::Deflate.new
    outputs << (z.deflate(content) << z.finish)
    z.close
  end
end

while thr.alive?
  begin
    thr.wakeup
  rescue ThreadError
  end
  Thread.pass
end
thr.join

p outputs.count { |output| Zlib::Inflate.inflate(output) == content }
  • zlib 3.2.3 gem: Zlib::BufError from deflate (Thread#wakeup leads to Zlib::BufError #57), 20 of 20 runs.
  • zlib 3.2.4 and master: Zlib::DataError from close, 20 of 20 runs. Same on Ruby 3.4.11's bundled zlib.
  • This branch: prints 100, 20 of 20 runs.

Cause

Since #88, each deflate() and inflate() call runs in its own rb_nogvl, and rb_nogvl handles pending interrupts when the call returns. zstream_run_func still stops its loop when the unblock function sets args->interrupt, and zstream_run_try then guesses whether to resume:

Fix

The loop no longer stops for an interrupt, and the resume block is removed. Interrupts still reach Ruby after each call through rb_nogvl. The args->interrupt flag and the unblock function are removed too, because nothing read the flag. rb_nogvl gets NULL as the unblock function.

Checked on this branch:

  • Thread#raise, Thread#kill, Timeout.timeout and a raising trap still stop deflate, inflate, Zlib.gzip and GzipWriter within about 60 ms.
  • With a Thread#wakeup storm or about 1,700 SIGUSR1 with a non-raising trap, every API gives correct output, with no slowdown: Deflate with each flush mode, Zlib.deflate, Zlib.gzip, GzipWriter, Zlib.inflate, chunked Inflate, Zlib.gunzip, and GzipReader#read and #readpartial.
  • The SIGINT/fork reproduction from Thread#wakeup leads to Zlib::BufError #57 passes 30 of 30 runs.

Test

TestZlibDeflate#test_thread_wakeup_interrupt deflates 256 KiB of random data 100 times in a thread that is woken up in a loop, then inflates every output. It runs in less than 0.5 s and is pended on TruffleRuby, like the existing interrupt tests.

  • Without the fix it fails on every run: Zlib::DataError from close. I checked this on macOS and on Linux (ruby 3.4.11, aarch64).
  • With the fix, bundle exec rake compile test passes on Ruby 4.0.6, 3.4.8, 3.3.10 and 3.2.9 (99 tests, 639 assertions on macOS). The existing test_thread_wakeup_interrupt tests in TestZlib and TestZlibGzipReader still pass.

Ruby's ruby_3_4 branch has #121, so it needs this change too.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The C-level change affects Ruby VM interrupt handling across all deflate and inflate paths and warrants final human review.

0 open findings

What changed in this PR

Fixes truncated deflate output when non-raising interrupts occur during stream processing.

Changes:

  • Continues zstream processing after interrupts until normal termination.
  • Adds regression coverage for repeated Thread#wakeup calls.
File Description
ext/​zlib/​zlib.c Removes interrupt-based loop termination and retry logic.
test/​zlib/​test_zlib.rb Verifies interrupted deflate streams remain complete.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Each deflate() and inflate() call runs in its own rb_nogvl since ruby#88, and
rb_nogvl handles pending interrupts when the call returns. So the loop
does not need to stop for an interrupt and resume.

The resume check from ruby#121 skipped the resume when avail_in == 0. Under
Z_FINISH, deflate can still have pending output then, so
Zlib::Deflate#finish could return before Z_STREAM_END and drop the end
of the stream.
@hsbt
hsbt force-pushed the fix-deflate-finish-after-interrupt branch from 6bb418f to 5538219 Compare November 9, 2026 04:46
@ioquatix

ioquatix commented Nov 9, 2026

Copy link
Copy Markdown
Member

I guess I'm okay with this change. I'm not sure if it's going to make interrupt handling worse? In the case that args->interrupt is no longer used, let's just get rid of it.

zstream_run_func no longer reads args->interrupt, so the unblock
function did nothing useful. Pass NULL instead: Ruby handles pending
interrupts when each deflate() or inflate() call returns.
@bruno-costanzo

Copy link
Copy Markdown
Author

Thanks @ioquatix. I removed args->interrupt and zstream_unblock_func. rb_nogvl and rb_thread_call_without_gvl now get NULL as the unblock function, because the unblock function did nothing else. Both paths in Ruby, and the fiber scheduler path, accept NULL.

About interrupt handling: each deflate() or inflate() call has its own rb_nogvl since #88, and its output is at most ZSTREAM_AVAIL_OUT_STEP_MAX (16 KiB). Ruby handles pending interrupts when each call returns. The flag only stopped the loop between two calls, and Ruby already handles the interrupt at the same point.

I checked it again with 64 MiB of random data at level 9:

  • Thread#kill and Thread#raise stop the deflate in less than 1 ms.
  • Timeout.timeout(0.2) stops it too.
  • A non-raising interrupt (Thread#wakeup, a trap that does not raise) no longer stops the stream. This is the fix.

bundle exec rake compile test: 110 tests, 0 failures, on Ruby 4.0.4.

One thing I saw while I checked, not related to this PR: on master, when Timeout or Thread#raise stops Zlib.deflate, the cleanup raises Zlib::DataError from deflateEnd, and that error replaces the original exception. I can open a separate PR for it if you want.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants