Ship precompiled gems for linux, musl and darwin - #157
Open
ThomasSevestre wants to merge 1 commit into
Open
ThomasSevestre wants to merge 1 commit into
ThomasSevestre wants to merge 1 commit into
Conversation
Installing zstd-ruby currently means compiling the bundled libzstd, which needs a toolchain and takes a while on every install. Cross-compile the extension with rake-compiler-dock instead and publish platform gems for x86_64/aarch64 linux (glibc and musl) and x86_64/arm64 darwin, each carrying binaries for Ruby 3.1 through 4.0. Anything outside that set - other platforms, Ruby 2.7 and 3.0, future Rubies - still installs the source gem and compiles as before. rake-compiler puts fat-gem binaries under lib/zstd-ruby/<X.Y>/ and generates no loader, so lib/zstd-ruby.rb now tries the ABI directory first and falls back to the flat path used by source installs. The native gems drop ext/ via the cross_compiling callback: the binaries are already there and the extension is cleared, so libzstd's sources are 2.2MB of dead weight. The gemspec also stops listing the "zstd" submodule gitlink, which is not a file. Six platforms times five ABIs is too heavy for every PR, so the workflow runs on version tags, on manual dispatch, and on PRs that touch the build files. Each gem is smoke-tested on a matching target - native runners for x86_64-linux and both macOS, QEMU arm64 containers for aarch64, Alpine for musl - before the tag build pushes it with trusted publishing. The source gem keeps going out through `rake release`, which is also what creates the tag. Verified by building the x86_64-linux and x86_64-darwin gems locally: the full spec suite passes against both when installed from the .gem, each Ruby loads its own ABI directory, and the darwin binary still exports only _Init_zstdruby, so the symbol isolation from SpringMT#104 survives cross-compilation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Installing zstd-ruby currently means compiling the bundled libzstd, which needs a toolchain and takes a while on every install. Cross-compile the extension with rake-compiler-dock instead and publish platform gems for x86_64/aarch64 linux (glibc and musl) and x86_64/arm64 darwin, each carrying binaries for Ruby 3.1 through 4.0.
Anything outside that set - other platforms, Ruby 2.7 and 3.0, future Rubies - still installs the source gem and compiles as before.
rake-compiler puts fat-gem binaries under lib/zstd-ruby/<X.Y>/ and generates no loader, so lib/zstd-ruby.rb now tries the ABI directory first and falls back to the flat path used by source installs.
The native gems drop ext/ via the cross_compiling callback: the binaries are already there and the extension is cleared, so libzstd's sources are 2.2MB of dead weight. The gemspec also stops listing the "zstd" submodule gitlink, which is not a file.
Six platforms times five ABIs is too heavy for every PR, so the workflow runs on version tags, on manual dispatch, and on PRs that touch the build files. Each gem is smoke-tested on a matching target - native runners for x86_64-linux and both macOS, QEMU arm64 containers for aarch64, Alpine for musl - before the tag build pushes it with trusted publishing. The source gem keeps going out through
rake release, which is also what creates the tag.Verified by building the x86_64-linux and x86_64-darwin gems locally: the full spec suite passes against both when installed from the .gem, each Ruby loads its own ABI directory, and the darwin binary still exports only _Init_zstdruby, so the symbol isolation from #104 survives cross-compilation.
You may need to register here to be able to publish pre compiled gems. https://rubygems.org/gems/zstd-ruby/trusted_publishers