Take the jpeg2000 release that carries 9/7 coefficients at float32 - #95
Merged
Merged
Conversation
THIS CHANGES PIXELS. 634 of the 3215 documents of the measured corpus render differently, by about 0.005 levels of 255. They move TOWARD the judge. Asked document by document -- for each page, which build lands closer to poppler -- float32 is closer on 413 of 612 and further on 171, with 28 tied. Under a 50/50 null over the 584 untied, 292 +/- 12 would be expected. That is what the reference predicts: poppler decodes JPEG 2000 through OpenJPEG, which works in float32, so carrying its precision carries its rounding. The conformance aggregate does NOT move: mean |diff| and the identical-pixel share are the same to four figures. That is the aggregate's resolution, not the absence of an effect -- it is a percentage over hundreds of millions of pixels and the change moves each page by 0.005 levels. The per-document count is what can see it. Decoding one 5868x7885 one-component scan alone, peak RSS falls 751 -> 567 MB and the live heap 540 -> 368 MB.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This changes pixels
go-images/jpeg2000v0.11.0 carries the inverse 9/7 coefficients atfloat32,which is the precision the reference carries them at — OpenJPEG reinterprets a
tile-component's
OPJ_INT32*asOPJ_FLOAT32*for 9/7 and holds no secondbuffer (
dwt.c:157,tcd.h:210).634 of the 3215 documents of the measured corpus render differently, by
about 0.005 levels of 255.
They move toward the judge, 2.4 to 1
The conformance aggregate does not move —
mean |diff|and theidentical-pixel share are the same to four figures on four of five populations.
That is the aggregate's resolution, not the absence of an effect: it is a
percentage over hundreds of millions of pixels, and each page moved by 0.005
levels. A number that does not move is not the same as a thing that does not
move.
Asked document by document instead — for each page, which build lands
closer to poppler:
612 judged. Under a 50/50 null over the 584 untied, 292 ± 12 would be expected;
413 is ten standard deviations away. Poppler decodes JPEG 2000 through
OpenJPEG, so carrying OpenJPEG's precision carries its rounding — the direction
is what the reference predicts.
The largest move either way is 0.004691 levels of 255. Nobody will see
this; it is not a fidelity improvement anyone can point at. It is a confirmation
that the precision is the right one, bought for a large amount of memory.
The memory, which is the point
Decoding one 5868×7885 one-component scan and nothing else, so the figures are
the decoder's:
99.92% of the JPEG 2000 streams in the scanned corpus use the 9/7 filter
(119 069 of 119 164, read from each codestream's COD marker), so this is on very
nearly every scanned page.
Not claimed
Speed. A float32 vector carries twice the lanes of a float64 one and
LiftStep97is vectorised, so this may also be faster — but it was notmeasured, because the machine was carrying two system daemons at 132% and 86% of
a core. A timing taken under that says more about the daemons than the code.
The module in the cache was diffed against the tree these figures were measured
on.
🤖 Generated with Claude Code