feat: Add protobuf diff and replay support with optional copy-builder tracking

[derektriley]

Description:
Adds generated protobuf diff and replay support for consumers such as Hiero’s block-stream state updates. When only a few fields change, consumers can transmit those fields and an explicit clear list, then reconstruct the complete value from the previous value.

Updates replace whole top-level fields. Nested messages, repeated fields, and maps are each replaced as a unit.

Generated model APIs

Every generated model receives these methods, including when copy-builder tracking is disabled. Signatures below use Model as the generated class name.

public static long diff(Model prior, Model next);
public static long diff(Model prior, Model next, long candidates);

Compute a mask of fields whose values differ. The first overload automatically uses copy-builder provenance when its origin is exactly prior, otherwise it compares every field. The second overload compares only the supplied candidates. Candidate fields are always checked by value, so calling a setter with an unchanged value does not create a difference.

public static Model partial(Model next, long mask);

Creates a value containing the selected fields, with all other fields reset to their defaults. Its protobuf encoding supplies the replacement fields.

public static int[] clearedFields(Model next, long mask);
public static int[] clearedFields(
        @Nullable Model prior, Model next, long mask);

Returns the protobuf field numbers to clear, in ascending order. An explicit clear list is necessary because default-valued fields are normally omitted from protobuf encodings.

For a selected oneof, the overload accepting prior clears only the previously active alternative when needed. Without prior, every inactive alternative is included.

public static boolean worthPartial(
        Model next, long mask, int minSaving, int overhead);

Checks whether omitted fields save at least minSaving + overhead bytes. It measures cheaper fields first and stops when the threshold is reached, avoiding a complete size calculation when possible. Consumers can use this to choose between partial and full updates.

public static Model applyPartial(
        Model prior, Model partial, int[] cleared);

Reconstructs the complete value directly from models, without serialization. It validates clear lists, rejects replace/clear conflicts and unknown clear numbers, and requires oneof switches to explicitly clear the old alternative.

Masks use declaration-order indexes, with one bit per ordinary field or oneof group. Models with more than 64 such entries use long[] instead of long throughout these signatures. Clear lists always use protobuf field numbers.

Byte-based replay

Adds these runtime methods to PartialApplier for consumers that retain serialized values:

public static Bytes splice(
        Bytes prior, Bytes partial, int[] cleared);

public static void validateCleared(int[] cleared);

splice() replaces complete protobuf field runs and removes cleared fields while preserving all other bytes. Nested payloads remain opaque, so replay does not require decoding and re-encoding messages.

Inputs must have ascending field order and canonical varints. The utility validates framing, ordering, and replace/clear conflicts; schema-specific validation remains the consumer’s responsibility.

Copy-builder tracking

Adds the generateCopyBuilderTracking compiler setting, disabled by default. It can be enabled to track provenance:

pbj {
    generateCopyBuilderTracking = true
}

When enabled, generated models implement CopyBuilderTracked<Model> and expose:

public @Nullable Model $copyBuilderOrigin();
public boolean $copyBuilderFieldChanged(int fieldNumber);
public long $copyBuilderChangedMask();
public Model $untracked();

$copyBuilderChangedMask() returns long[] for wide models.

Copy-builder chains retain their original untracked instance and accumulate a mask of fields touched by setters. This accelerates comparisons against that exact origin. Comparisons against another instance fall back to checking every field.

Tracking metadata is excluded from equality, hashing, text output, and codecs. $untracked() removes shallow provenance before long-lived storage; the runtime also provides CopyBuilderTracked.untracked(value) for generic callers.

Comparison semantics and validation

Generated models also implement DiffSupport.Value.$equalsForDiff(Object). DiffSupport.equal(Object, Object) supplies comparison support for nested generated messages, collections, oneofs, and raw floating-point payloads without changing ordinary Object.equals().

Top-level unknown fields require a full replacement. Tests cover tracking chains and builder reuse, wide masks, restored values, explicit presence, oneof switches, enum ordinals, NaN payloads, savings thresholds, and both object-based and byte-based replay.

Focused compiler/runtime test targets passed, along with 97 integration checks.

Related issue(s):

Fixes #

Notes for reviewer:

Checklist

• Documented (Code comments, README, etc.)
• Tested (unit, integration, etc.)

[github-actions]

JUnit Test Report

   528 files  + 7     528 suites  +7   30s ⏱️ -2s
 1 573 tests +36   1 569 ✅ +36   4 💤 ±0  0 ❌ ±0 
10 815 runs  +60  10 787 ✅ +60  28 💤 ±0  0 ❌ ±0 

Results for commit d8fafa4. ± Comparison against base commit c1b9317.

This pull request removes 6 and adds 42 tests. Note that renamed tests count towards both.

com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [1] FLOAT, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x000000001249f730@61a17f3, [0.1, 0.5, 100.0], 12, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x000000001249f958@352d3749
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [1] STRING, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000000124add40@60e5b3a8, [string 1, testing here, testing there], com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000000124adf68@1e084a26
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [2] BYTES, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000000124ae190@2703785e, [010203, ff7f0f, 42da07370bff], com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000000124ae3b8@5a2297d7
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [2] DOUBLE, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x000000001249fb80@5d3d5242, [0.1, 0.5, 100.0, 1.7653472635472653E240], 32, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x000000001249fda8@1a5c0b8
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [3] BOOL, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000000124a4000@3f2feab7, [true, false, false, true, true, true], 6, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000000124a4228@8b71052
com.hedera.pbj.runtime.ProtoWriterToolsTest ‑ [4] ENUM, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000000124a4450@6d7d0946, [0, 2, 1], 3, com.hedera.pbj.runtime.ProtoWriterToolsTest$$Lambda/0x00000000124a4678@2dc72c0c

com.hedera.pbj.compiler.CopyBuilderGenerationTest ‑ trackingIsOffByDefaultAndCanBeTurnedOn()
com.hedera.pbj.runtime.PartialApplierTest ‑ [10] "1100"
com.hedera.pbj.runtime.PartialApplierTest ‑ [11] "1a05ff"
com.hedera.pbj.runtime.PartialApplierTest ‑ [12] "1affffffffffffffffff01"
com.hedera.pbj.runtime.PartialApplierTest ‑ [13] "2500"
com.hedera.pbj.runtime.PartialApplierTest ‑ [14] "10010801"
com.hedera.pbj.runtime.PartialApplierTest ‑ [15] "808080801001"
com.hedera.pbj.runtime.PartialApplierTest ‑ [16] "808080808000"
com.hedera.pbj.runtime.PartialApplierTest ‑ [1] "00"
com.hedera.pbj.runtime.PartialApplierTest ‑ [2] "04"
…

♻️ This comment has been updated with latest results.

[github-actions]

Integration Test Report

    432 files  +  4      432 suites  +4   20m 34s ⏱️ + 6m 41s
115 163 tests +115  115 163 ✅ +115  0 💤 ±0  0 ❌ ±0 
115 407 runs  +115  115 407 ✅ +115  0 💤 ±0  0 ❌ ±0 

Results for commit d8fafa4. ± Comparison against base commit c1b9317.

This pull request removes 2 and adds 117 tests. Note that renamed tests count towards both.

com.hedera.pbj.integration.test.ParserNeverWrapsTest ‑ [1] com.hedera.pbj.integration.test.ParserNeverWrapsTest$$Lambda/0x0000000072c96858@452f1f33
com.hedera.pbj.integration.test.ParserNeverWrapsTest ‑ [2] com.hedera.pbj.integration.test.ParserNeverWrapsTest$$Lambda/0x0000000072c96aa0@c261378

com.hedera.pbj.integration.test.CopyBuilderTrackingTest ‑ chainsRetainTheRootAndSettersMarkCandidatesEvenWhenValuesAreRestored()
com.hedera.pbj.integration.test.CopyBuilderTrackingTest ‑ copyBuildersAndUntrackedCopiesPreserveUnknownFields()
com.hedera.pbj.integration.test.CopyBuilderTrackingTest ‑ messageBuilderListVarargsMapsAndOptionalWrapperSettersAreTracked()
com.hedera.pbj.integration.test.CopyBuilderTrackingTest ‑ oneofSettersAndClearMarkEveryAlternativeIncludingSelectedZero()
com.hedera.pbj.integration.test.CopyBuilderTrackingTest ‑ trackingHasNoEffectOnValueSemanticsCodecsOrCachedModels()
com.hedera.pbj.integration.test.CopyBuilderTrackingTest ‑ wideMasksAreImmutableSnapshotsAcrossBuilderReuseAndForks()
com.hedera.pbj.integration.test.ParserNeverWrapsTest ‑ [1] com.hedera.pbj.integration.test.ParserNeverWrapsTest$$Lambda/0x0000000064cc4cf8@32bf8920
com.hedera.pbj.integration.test.ParserNeverWrapsTest ‑ [2] com.hedera.pbj.integration.test.ParserNeverWrapsTest$$Lambda/0x0000000064cc4f40@6d6112c2
com.hedera.pbj.integration.test.PartialDiffTest ‑ com.hedera.hapi.node.state.addressbook.tests.NodeTest
com.hedera.pbj.integration.test.PartialDiffTest ‑ com.hedera.hapi.node.state.blockrecords.tests.BlockInfoTest
…

♻️ This comment has been updated with latest results.