Skip to content

ci: download TCK snapshots from the serialization_test_data layout - #775

Open
leerho wants to merge 1 commit into
mainfrom
tck-test-data-layout
Open

leerho wants to merge 1 commit into
mainfrom
tck-test-data-layout

Conversation

@leerho

@leerho leerho commented Oct 5, 2026

Copy link
Copy Markdown
Member

Updates tools/download_serialization_test_data.sh for the new TCK layout and moves its pin to the current TCK main.

Changes

  • New TCK layout. The TCK moved its snapshots from serialization/<lang>/snapshots/ to serialization_test_data/<lang>_generated_files/ (Use the serialization_test_data/<lang>_generated_files layout datasketches-tck#14), the same layout this repository uses. The script now matches the new archive paths and uses --strip-components=3 instead of 4.
  • Pin. TCK_REVISION moves from d363b12 (July) to e748b0a, the current TCK main.
  • serde-compat.yml. The Java matrix entry passed java to the download script, which only accepts cpp and go, so that job failed. The check_java_files profile generates its own snapshots before checking them, so the download step is now skipped for that entry.

Verification

  • With no arguments, the script extracts 202 C++ and 253 Go snapshots from TCK e748b0a.
  • mvn test -P check_go_files passes.
  • mvn test -P check_java_files passes.
  • mvn test -P check_cpp_files has one known failure: AodSketchCrossLanguageTest.checkCpp, "Incompatible Seed Hashes. 0, 93cc".

About the known failure

It's a real C++/Java incompatibility that the old pin hid. It is not caused by this change.

  • Report a zero seed hash for an empty compact sketch datasketches-cpp#518 made empty compact theta and tuple sketches write a seed hash of 0, to match Java's EmptyCompactSketch. In C++, array-of-doubles (AoD) is built on the generic tuple sketch, so empty AoD sketches now write 0 too.
  • Java's AoD sketches write the real seed hash for an empty sketch, and their readers reject 0.
  • The TCK picked up that C++ change on Sep 1, after the old pin d363b12, so this test never saw it until now.

Standard mvn clean test is unaffected. The failure only shows up when the cross-language profiles run.

🤖 Generated with Claude Code

The TCK moved its snapshots from serialization/<lang>/snapshots to
serialization_test_data/<lang>_generated_files (apache/datasketches-tck#14),
the same layout this repository uses. Match the new archive paths, strip
one fewer path component, and pin the TCK at e748b0a, its current main.

The serde-compat workflow's Java entry passed `java` to the download
script, which only accepts cpp and go. The check_java_files profile
generates its own snapshots before checking them, so skip the download
for that entry instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leerho
leerho requested review from proost and tisonkun October 6, 2026 18:36

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant