[roodboi]

Follow-up to #377 (HACK-1333, merged as 6b2bd73) for HACK-1389. #377 made the application benchmark protocol checker refuse any number outside ±(2⁵³ − 1). Review found four gaps in that work.

Root cause and fix

1. The documentation and the code disagreed about early refusals. The page says an out-of-range number is refused as summary_number_out_of_range "alongside any code for the field that holds it". But validate_summary returned early on summary_revision_mismatch and summary_workload_unknown before it scanned any numbers, so the number code never appeared next to those codes. I changed the code to match the page: the range scan now runs first, and both early returns include its result. For example, a summary with protocol_revision: 2 and a NaN now reports ["summary_number_out_of_range", "summary_revision_mismatch"]. The page now names this case explicitly.

2. A repeated key hid a number from the scan. json.loads keeps only the last value of a repeated key. The frozen protocol with , "notes": NaN, "notes": 1 appended was reported valid (exit 0) because the NaN never reached the scan. _load now uses an object_pairs_hook that refuses any repeated key at any depth, with two new closed codes: protocol_key_duplicate and summary_key_duplicate (exit 2, before any other check). I-JSON (RFC 7493), which the page already cites for the number range, forbids repeated member names too.

3. One input gave a different code depending on the Python version. A 5000-digit integer literal was refused as protocol_number_out_of_range on 3.9 and as protocol_unreadable on 3.11 and later. Since 3.11, Python refuses to convert an integer of more than 4300 digits, and the resulting ValueError was caught as an unreadable file. Any literal with more digits than MAX_JSON_NUMBER (16) is out of range whatever its value, so a parse_int hook now parses it as ±(MAX_JSON_NUMBER + 1). The result is still an int, so every type check behaves as it does for the exact value, and the code is now protocol_number_out_of_range on every version. The same limit also crashed the in-process API on 3.11+: validate_protocol raised ValueError from _same (json.dumps) for a 5000-digit host.logical_cpus. _same now treats a value that cannot be serialized as different, which keeps the docstring's "never raises on any JSON value" true.

4. No test tied the page to the code. A new test reads every ±(2ⁿ − 1) bound on the page and checks that it equals MAX_JSON_NUMBER. It also checks that the page contains no ±( written any other way, and that each related code name is both in DIAGNOSTICS and on the page.

Stdlib only, as before.

Tests

New tests in tests/python/test_application_benchmark_protocol.py:

• test_an_out_of_range_number_is_reported_with_an_early_refusal: covers NaN, 2**53 and -10**400, each with a revision or workload refusal, both in process and through the CLI. A 2**53 revision gets both codes.
• test_a_repeated_key_is_refused_so_no_number_is_shadowed: the exact repro from the issue, a nested repeated id in a lane, and a repeated key in a summary. Each has a passing control that has no repeated key.
• test_integer_literals_of_any_length_get_the_range_code_on_every_interpreter: integer literals of 5000 digits (both signs), 2⁵³ and 10¹⁶ in a protocol and in a summary. ±MAX_JSON_NUMBER, 0 and -0 stay valid. A 5000-digit literal in a field the checker reads gets the same codes as an exact out-of-range value there.
• test_an_integer_beyond_the_conversion_limit_never_raises_in_process
• test_document_states_the_number_bound_and_its_codes

Evidence

• The repro on the base commit 7d9c89f, on both interpreters: the repeated-key protocol was valid/exit 0; the 5000-digit literal was protocol_number_out_of_range on 3.9.6 and protocol_unreadable on 3.14.8; a revision-2 summary with NaN was ["summary_revision_mismatch"] only. After the fix, both interpreters report protocol_key_duplicate, protocol_number_out_of_range and ["summary_number_out_of_range", "summary_revision_mismatch"].
• The checker validates the frozen protocol, and all 45 tests in the file pass, on /usr/bin/python3 (3.9.6) and the mise Python (3.14.8). Hosted CI runs this step on 3.12.
• Mutation proof: I applied ten mutants one at a time, and each was killed by the new tests. They cover each early return dropping the number code, removing the repeated-key hook, removing the parse_int hook, setting the digit bound one too small, _same no longer catching ValueError, MAX_JSON_NUMBER changing to 2**54 - 1, the page's bound changing to 2⁵⁴, the page's bound written as 2^53 - 1, and a code name losing its backticks on the page. The parse_int and _same mutants fail only on 3.11 and later, which is expected because 3.9 has no conversion limit. Setting the digit bound one too large survives, but that mutant is equivalent: 17-digit integers are still out of range when parsed exactly.
• Gates: root bun run check (includes privacy:check) passes; bun run --cwd packages/cli typecheck has 0 errors; gitleaks 8.30.1 over origin/next..HEAD finds no leaks.
• Hosted CI at 8465ff7: the step for this checker ("Check application benchmark protocol controls", Python 3.12, in Runtime state models) passed, as did every docker-e2e shard, Candidate core (ubuntu-latest), the config compilers, secret-scan, runtime-images and linux-process-lifetime.
  • test (full Bun suite): passed on run attempt 2 at the exact head 8465ff7 (job 120494444599, 54m48s). On attempt 1, 8211 of 8212 passed. The one failure was tests/native-compose-storage-witness-xattr-owner.test.ts, "same-birth root replacement while awaiting the final reader cannot publish ready from its pinned old directory", which timed out at 5000 ms with an unhandled E_NATIVE_COMPOSE_STATE between tests. That test is not in the known-flake list, and this change does not touch it. The rerun was not started by this workflow unit.
  • Candidate core (macos-latest), "Test candidate optional feature contracts", still fails (not rerun by this unit, because it is not a listed known flake): hostname_authority::managed tests authority_startup_waits_for_inspection_before_reading_pool_state (provider_busy) and authority_startup_contention_deadline_admits_no_pool_or_socket_effect (NotFound). This PR changes no Rust. The likely cause is a fixture race: the temp root in packages/runtime-core/src/provider/hostname_authority/managed.rs is pid + SystemTime nanos with no counter, and macOS SystemTime has microsecond resolution, so two parallel tests can share one root. The same job passed on next at 7d9c89f and on every other recent CI run checked.
• Re-verified locally at 8465ff7: all 45 tests pass on /usr/bin/python3 (3.9.6) and the mise Python (3.14.8). Through the CLI, the repeated-key repro gives protocol_key_duplicate and the 5000-digit literal gives protocol_number_out_of_range on both, each with exit 2.

Gaps

• A repeated key is refused on its own, before the number scan and other checks run, so a file with both problems reports only *_key_duplicate. That is deliberate: the parsed document is ambiguous.
• A top-level non-object still reports only protocol_not_object / summary_not_object, without a number code. The page's guarantee covers numbers inside a protocol or summary object.
• The in-process API (validate_protocol, evaluate) takes already-parsed values, so it cannot see repeated keys. Only the CLI's _load refuses them.
• No CLI surface or consumer agent guidance changed, so docs:cli-reference and generate:agent-plugins are not needed.

Refs HACK-1389

🤖 Generated with Claude Code

[linear-code]

HACK-1389