Repository navigation
Conversation
…numerators Doubling denom until it exceeds numer overflows to zero once numer >= 2^63, so the loop never terminated. ub_on_num_levels(n) calls it with the merged stream weight, which made a merge reaching n >= 2^63 hang. Compare against numer / 2 instead, which is equivalent and cannot overflow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
update(item, weight) is equivalent to calling update(item) weight times. A weight that fits into the free space of level zero is applied as plain updates; a larger one builds an exact sketch holding one copy of the item at each level whose bit is set in the weight and merges it, so the cost is logarithmic in the weight. This mirrors the weighted update in datasketches-java. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
kll_sketch::update(item, weight), equivalent to callingupdate(item)weighttimes. This mirrorsupdate(item, weight)on the KLL sketches in datasketches-java.Behavior
weightis smaller than the free space in level zero, the item is insertedweighttimes as plain updates. No compaction can happen on this path.hcounts as2^h, the exact sketch holds one copy of the item at each level whose bit is set inweight. Bits above level 60 fold intoweight >> 60copies at level 60, because level capacities are only defined up to 61 levels. This path costs one merge with a sketch of at most 75 items, however large the weight.std::invalid_argument, the same as in Java. NaN items are ignored, as inupdate(item).Fix included
The first commit fixes an infinite loop in
kll_helper::floor_of_log2_of_fraction. Oncenumer >= 2^63, doublingdenomoverflows to zero before it exceedsnumer.ub_on_num_levels(n)calls this function with the merged stream weight, so any merge that reachesn >= 2^63used to hang. A weighted update makes that weight easy to reach. The fix comparesdenomagainstnumer / 2, which gives the same result without the overflow. The fix has its own unit cases.Tests
New sections in
kll_sketch_test.cpp:2^40 + 12345weightUINT64_MAX - 1weight, with a serialization round tripThe kll test suite passes on clang (macOS arm64) in a plain debug build and with
-fsanitize=address,undefined.Companion PR for datasketches-rust: apache/datasketches-rust#293
🤖 Generated with Claude Code