Skip to content

FFT3DFilter: plan with FFTW_ESTIMATE so preview = encode across processes (task #80) - #234

Merged
StuartCameronCode merged 9 commits into
mainfrom
stuart/80-windows-ci-fft3dfilter-determinism-test
Nov 20, 2026
Merged

StuartCameronCode merged 9 commits into
mainfrom
stuart/80-windows-ci-fft3dfilter-determinism-test

Conversation

@StuartCameronCode

Copy link
Copy Markdown
Owner

Fixes the Windows push-gate failure of fft3dfilter is deterministic (task #62) (seen on PR #229, run 44510129664): a second vspipe process hashed differently from the first.

Cause

FFT3DFilter's default measure=1 (FFTW_MEASURE) times FFTW's candidate plans when the filter is created and keeps the fastest, which depends on machine load. Plans round differently, so a preview and its encode (two processes) could filter differently. Probed in 20 fresh processes per value on hosted runners: with measure=1 some call varied on every x86 runner (Windows, Linux, Intel Mac); Apple Silicon never did, which is why task #62 missed it. With measure=0 (FFTW_ESTIMATE): one hash per call everywhere.

FFTW also reuses a measured plan made earlier in a process for a later ESTIMATE request of the same problem, so passing measure=0 at our call sites is not enough on its own while havsfunc (QTGMC, MCTemporalDenoise) passes none.

Change

  • Template: all three FFT3DFilter calls pass measure=0: the Noise Reduction FFT3D method and TemporalDegrain2's spatial and post passes. test_261 fails on any template call without it; test_96 asserts it.
  • Deps patch: fft3dfilter-r2-determinism.patch also makes measure=0 the plugin's default, published as deps 1.18.0-rc2 (pre-release, not Latest). deps-version.json points at it.
  • Tests:
    • The push-gate determinism test passes measure=0 on every class and adds td2_post (48 px blocks). From rc2 it also covers the bare default and havsfunc's QTGMC, and asserts the default equals explicit measure=0.
    • A second push-gate test reads the patch to confirm the default changed.
    • The heavy preview = encode test gains a QTGMC fft3dfilter case.
  • Docs: a report at docs/reports/task-80-fft3dfilter-fftw-estimate (with probe scripts and raw data), an ENGINEERING_NOTES entry, a reports index row, CLAUDE.md's FFT3DFilter rules ("never measure=1"), and a correction note on the task perf(arm64): route std.Expr through akarin's LLVM JIT #62 report.

Cost

Only TemporalDegrain2's post pass, which is off by default: that call is 15–32% slower on x86 (~11–15% on an M1 Pro), and a whole TD2 job with it 5–23% slower. Its picture moves ≤ 1 level in < 0.1% of 8-bit samples. Everything else is within speed noise.

Testing

  • cargo test: test_96, test_261 pass.
  • Push gate (ci-test.yml) on the rc2 bundle, run 44710870617: green on Linux x64, macOS arm64, macOS x64 and Windows. Both fft3dfilter tests ran with the rc2-only checks.
  • deps 1.18.0-rc2 assets were checked before publishing: each zip against its sidecar SHA-256, version.json says rc2, and each v2 delta's baseSha256 names its bundle.
  • Not yet run: the heavy integration_fft3dfilter_determinism_test.dart against rc2 (runs with nightly).

🤖 Generated with Claude Code

StuartCameronCode and others added 9 commits November 20, 2026 19:24
…orkflow

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ython

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FFT3DFilter's default measure=1 has FFTW time its candidate plans once per
process; different plans round differently, so a preview and its encode (two
vspipe processes) could filter differently. The Windows push gate's
second-process check caught it on some runners (td2_spat; 5 of 8 classes in
another run).

- Template: the Noise Reduction FFT3D method and both TemporalDegrain2
  passes pass measure=0. test_261 fails on any template FFT3DFilter call
  without it; test_96 checks the generated script.
- Patch: measure defaults to 0, for havsfunc's calls (QTGMC,
  MCTemporalDenoise), which pass none, and because FFTW reuses a measured
  plan made earlier in a process for a later ESTIMATE request of the same
  problem (measured on macOS arm64). Needs deps 1.18.0-rc2.
- Push gate: every class passes measure=0; adds TD2's post-pass class and,
  from 1.18.0-rc2, the plugin's default and havsfunc's QTGMC call; a second
  test reads the patch for the default. _depsVersionBefore compares
  pre-releases (rc1 < rc2 < release).
- Heavy: a QTGMC fft3dfilter preview = encode case (from 1.18.0-rc2).
- probe-plugin-formats.py: the template's arguments (format table unchanged).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ter only, interleaved

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…18.0-rc2 build artifacts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the probe workflow

- docs/reports/task-80-fft3dfilter-fftw-estimate: CI evidence, the
  cross-process probe on 10 hosted runners x 4 runs (rc1 and the rc2
  build), the wisdom-inheritance finding, speed and picture movement;
  tables generated from data/ by scripts/tables.py + fill.py.
- Task #62's report gets a dated note and a corrected decision (left
  visible); ENGINEERING_NOTES entry; reports index row; CLAUDE.md rule
  ("Never measure=1") and the 1.18.0 rc2 history row.
- The push-gate determinism test gets a 5-minute timeout: with the rc2
  classes (default, QTGMC) it ran past the 30 s default on the 4-core
  Linux runner (CI run 44696475156); Windows and macOS passed.
- The temporary probe workflow is removed (its runs are recorded in the
  report).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- docs/reports/task-80-fft3dfilter-fftw-estimate: CI evidence, the
  cross-process probe on 10 hosted runners x 4 runs (rc1 and the rc2
  build), the wisdom-inheritance finding, speed and picture movement;
  tables generated from data/ by scripts/tables.py + fill.py.
- Task #62's report gets a dated note and a corrected decision (left
  visible); ENGINEERING_NOTES entry; reports index row; CLAUDE.md rule
  ("Never measure=1") and the 1.18.0 rc2 history row.
- The push-gate determinism test gets a 5-minute timeout: with the rc2
  classes (default, QTGMC) it ran past the 30 s default on the 4-core
  Linux runner (CI run 44696475156); Windows and macOS passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…0 default)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@StuartCameronCode
StuartCameronCode merged commit 5b5a84e into main Nov 20, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant