Skip to content

perf: テストとCIおよび比較パイプラインの重複処理を最適化する - #1

Open
zredjet wants to merge 3 commits into
masterfrom
perf/optimize-tests-and-ci
Open

zredjet wants to merge 3 commits into
masterfrom
perf/optimize-tests-and-ci

Conversation

@zredjet

@zredjet zredjet commented Nov 15, 2026

Copy link
Copy Markdown
Owner

概要

全 2,375 件のテスト品質・カバレッジ・検出条件・予算制約を一切維持したまま、テスト内部・CI・製品比較パイプラインの重複処理を解消して実行時間を最適化します。

主な変更点

  • Step 1(テスト内部キャッシュ):
    • GoldenData に UnoptimizedDifference と PNG/JSON キャッシュを追加し、42 ゴールデンケースに対する未最適化パス(useGroupBounds: false)の重複計算(3回→1回)を解消
    • PageFlowCliTests・AnchoredAcceptanceTests の固定 Fixture SHA-256 検証および carry_enabled: false(off)基準比較のキャッシュ化・--no-html 指定
  • Step 2(CliProcess.Run のインプロセス化):
    • CliProcess.Run を CliApplication.Run のインプロセス呼び出しへ移行し、約 154 回の dotnet サブプロセス起動オーバーヘッドを削減
    • 実プロセス起動・標準入出力の UTF-8 配線・タイムアウト強制終了を検証する 6 件は CliProcess.RunProcess で実サブプロセス実行を維持
  • Step 3(macOS ランタイム NuGet の CI キャッシュ化):
    • tools/build-macos-runtime.py に --skip-existing(既存 .nupkg の整合性検証付きスキップ)を追加
    • .github/workflows/ci.yml で out/packages/ReportDiff.OpenCvSharp4.runtime.osx.*.nupkg を actions/cache でキャッシュ化
  • Step 4(製品比較パイプラインの重複解消):
    • ComparisonInput.ReadPage に最大 4 ページの LRU キャッシュを導入し、同一ページの PDFium 再ラスタライズを排除
    • PdfTextReader のページ解析・単語抽出(最大 4 ページ LRU)・ReadRowWords・InspectFonts、および PdfFontInspector の IndirectReference 単位のフォント検査結果をキャッシュ化
    • AnchoredRun.Prepare の AnchoredComparisonResult を AnchoredRun.Save で再利用し、同一内容面の二重比較を解消
    • PageFlowRun.Prepare で算出した AlignmentResult・PageFlowComparison・基準 PageComparison を ComparisonRunner / RowComparer へ引き継ぎ、PageFlowPlan 再構築時の未変更帯検証・比較面を再利用
    • MatBuffers・PageFlowBandVerifier.Digest・PageFlowCollector で連続メモリの一括コピーおよび白行ハッシュ再利用を導入

ローカル検証結果

  • dotnet build -c Release: 警告 0・エラー 0
  • dotnet test --no-build --no-restore -c Release --parallel none --minimum-expected-tests 2375 --progress on:
    • 合計 2,375 件 / 成功 2,375 件 / 失敗 0 件 / スキップ 0 件
    • ローカル実行時間: 9 分 33 秒(573.3 秒)→ 6 分 32 秒(392.5 秒)(約 31.5% 短縮)

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T06:42:55.201059Z abfd58d Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e39931d9de

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/ReportDiff.Cli/PageFlowRun.cs Outdated
Comment on lines +89 to +90
if (baseline is not null) run.baselines.Add(page.PageNumber, baseline);
if (comparison is not null) run.comparisons.Add(page.PageNumber, comparison);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Bound the retained page-flow comparison buffers

For multi-page PDFs whose flow plan is ready, this loop now retains every page's baseline and flow comparison until adoption has finished for the entire document. PageFlowComparison owns two BGR content images, content masks, and display masks, while the baseline owns additional masks; previously these buffers were disposed after each page's adoption check. For approximately full-page content surfaces, 30 A4 pages at 300 DPI can therefore retain roughly 3 GB even without optional removal masks or projection data, before working buffers and the new raster cache. Such inputs remain within MaximumPixels, so comparisons that previously ran page-by-page can now exhaust memory. Use a byte-bounded cache and dispose/recompute evicted comparisons rather than retaining the whole document.

Useful? React with 👍 / 👎.

- PageComparer.Cluster: fast-path 0px raw difference masks before array copy/allocation and replace LINQ byte count with SIMD Span.Count
- RowGroupValidator & RowSupport: replace per-pixel HashSet/LINQ delegates with flat array/loop helpers
- PageFlowCliTests & AnchoredAcceptanceTests: cache repeated 'on' CLI runs and share rendered fixture pages/text across test cases
- Bound retained PageFlowComparison and PageComparison buffers in PageFlowRun to 256 MiB via RetainedBytes accounting, disposing overflow entries and falling back to on-demand comparison
- Immediately clear retained PageFlowComparison entries when any page fails flow adoption
- Release temporary ProjectionData after RowProjection.Create in PageFlowComparison.Create
@zredjet

zredjet commented Nov 15, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: abfd58d0c2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant