Skip to content

Do not write through a shared buffer in GzipReader#read and #readpartial - #190

Open
aristocratte wants to merge 1 commit into
ruby:masterfrom
aristocratte:fix-gzipreader-outbuf-shared
Open

aristocratte wants to merge 1 commit into
ruby:masterfrom
aristocratte:fix-gzipreader-outbuf-shared

Conversation

@aristocratte

Copy link
Copy Markdown

Zlib::GzipReader#read(len, buf) and #readpartial(len, buf) overwrite earlier copies of the buffer.

require "zlib"
require "stringio"

gz = Zlib::GzipReader.new(StringIO.new(Zlib.gzip("A" * 1000 + "B" * 1000)))
buf = "".dup
copy = gz.readpartial(1000, buf).dup   # keep the first chunk
gz.readpartial(1000, buf)              # read the next one into the same buffer
copy[0, 4]
# expected: "AAAA"
# actual:   "BBBB"

A long String and its dup share one allocation until one of them is modified. zstream_shift_buffer() fills the buffer with rb_str_resize() followed by memcpy(), and rb_str_resize() returns at once for a shared string when the length does not change. The write then goes through the shared memory, so a frozen copy, or the copy a Hash made of its key, is overwritten too.

rb_str_resize() also clears the coderange only when the string shrinks, so the buffer kept the coderange of its previous contents: after reading "\xFF" * 10 into a buffer that was "".dup, buf.ascii_only? was true.

It happens when the buffer holds a string too long to be embedded (623 bytes or more on Ruby 4.0), the next read returns the same number of bytes, and those bytes are already decompressed in the internal buffer. When GzipReader has to read from the underlying IO during the call, that IO's readpartial unshares the buffer first.

This change calls rb_str_modify() on the buffer before writing into it, as the other methods that read into a buffer do.

Reproduced with zlib 3.2.4 (this repository), 3.2.3 (Ruby 4.0.9 and 3.4.12) and, for readpartial, 3.1.2 (Ruby 3.3.13). The two new tests fail without the change; with it the suite passes (114 tests, 608 assertions) on Ruby 4.0.3.

Reported through HackerOne, where it was handled as a regular bug.

🤖 Generated with Claude Code

zstream_shift_buffer() fills the buffer given by the caller with
rb_str_resize() followed by memcpy().  rb_str_resize() returns at once
for a string that shares its memory when the length does not change,
and it only clears the coderange when the string shrinks.

So when the buffer still shared its memory with a copy (String#dup, a
frozen copy, the copy a Hash makes of its key), the next read of the
same length overwrote that copy, and the buffer kept the coderange of
its previous contents.

    gz = Zlib::GzipReader.new(StringIO.new(Zlib.gzip("A" * 1000 + "B" * 1000)))
    buf = "".dup
    copy = gz.readpartial(1000, buf).dup
    gz.readpartial(1000, buf)
    copy[0, 4] # => "BBBB"

Call rb_str_modify() on the buffer before writing into it.

Co-Authored-By: Claude Mythos 5.2 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant