feat(storage): answer get-or-closer requests (ADR-0020)

[mickvandijke]

ADR-0020: a chunk GET returns the chunk, or closer peers, in one request, as Kademlia's FIND_VALUE does. This PR adds the ADR and the node side.

Why. On production, a native chunk read spends 82% of its time finding a holder, because the closest-peers lookup runs to convergence before the GETs start. Asking each queried peer for the chunk as well as for closer peers cuts native p50 from 5.95 s to about 1 s; see the investigation.

Node.

• A node that holds the chunk answers GetOrCloserRequest with Found.
• Otherwise it answers Closer, with saorsa-core's encoding of the FIND_NODE answer it would give the authenticated requester.
• A node with storage enabled announces get-or-closer/1 in its user agent. One with storage off never answers chunk requests, so it does not announce it.
• Get-or-closer gets its own rows in the chunk traffic summary (group 4), and each answer is logged as get_or_closer_rpc.
• Browsers are unchanged. The WebRTC Direct listener still refuses the request, and HELLO does not list it. A browser variant was built and A/B'd on the dev testnet, and was not faster there (ant-client#221), so it is not part of this stack. The ADR records it and when to revisit it.

Compatibility by design.

• Old client, new node: the request is never sent; nothing changes.
• New client, old node: the client sees no token, so it sends FIND_NODE and a GET in the same step instead. That works with every deployed node at the same speed.
• Lookup finds no holder: the client falls back to today's read path, so the absence rule is unchanged.

Stack, merge in this order:

1. feat(dht): expose lookup steps for caller-driven lookups saorsa-core#173
2. feat(chunk): add a get-or-closer request ant-protocol#44
3. this PR
4. feat(read): read chunks through a get-or-closer lookup (ADR-0020) ant-client#221

This PR pins saorsa-core and ant-protocol at those branches by git rev.

Linear issue

Closes V2-1358

Risk tier

• T0 — docs / tooling / CI / pure UX-output. Repo CI only.
• T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
• T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
• T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Compatibility

• Wire: the node answers the new GetOrCloserRequest, and its user agent gains the token get-or-closer/1. The node/ prefix is kept. Every existing message is unchanged.
• Storage: none.
• API: none outside the crate. ChunkRequestContext::with_requester and node::user_agent(serves_chunks) are crate-private.

Semver impact

• breaking
• feature
• fix

Test evidence

• cargo test --lib --features test-utils: 1,168 passed, after merging main's LMDB store removal (refactor(storage)!: remove the LMDB chunk store, and never refuse a start over what it left behind #218). New tests:

  • Found for a held chunk
  • an error rather than an empty answer without a routing table
  • the user agent announces the token only when serving chunks, and keeps the node/ prefix

• cargo test --test e2e --features test-utils test_get_or_closer_returns_chunk_or_closer_peers: passed. On a local test network, the holder answers Found, and a non-holder answers Closer whose peers the client decodes as a FIND_NODE answer, without the requester.

• cargo clippy --all-features -- -D clippy::panic -D clippy::unwrap_used -D clippy::expect_used -D warnings: clean.

• cargo +1.99.0 clippy --all-targets --all-features -- -D warnings, as CI runs it: clean.

• Live. A client from the ant-client PR downloaded a 612 MB production file in 56 s, against 242 s for the current client, with identical bytes. All 150 chunks came through the new lookup, on the route for nodes without the token.

• Dev testnet comparison against main, matched (2026-10-09, native clients). Two 60-node fleets on the same layout:

  • 30% symmetric NAT across 9 regions.
  • 4 uploaders (50 MiB) and 2 verified downloaders per fleet, over a 60-minute window.
  • Both fleets on the rc-2026.10.1 release baseline. The clients differ only by the ADR-0020 commits.
  • Downloads paced to one every 40 s per downloader, so both fleets serve the same reads.

  	main	this stack	delta
  download p50 / p95 / max	20.5 / 28.6 / 35.5 s	9.7 / 15.5 / 20.4 s	−53% / −46% / −43%
  downloads ok / failed	176 / 0	176 / 0	same load
  read bytes nodes sent per byte downloaded	1.35	1.73	duplicate copies
  no-merkle uploads ok, p50 / p95	211, 32.9 / 45.6 s	211, 33.2 / 45.1 s	identical
  merkle uploads ok, p50 / p95	130, 54.1 / 66.2 s	112, 62.9 / 76.1 s	−14%, +16% / +15%
  node CPU mean	27.2%	26.3%
  node restarts	0	0

  • The routing works on a real network: since the redeploy, this stack's nodes served only get-or-closer requests (127 before T0), and main's only plain GETs.
  • The merkle upload gap is not attributed to this change:
    • It does not touch the merkle payment, quote or PUT path.
    • Non-merkle uploads, which run the same lookups and PUTs, are identical.
    • The kad fleet's chain (Anvil) droplet showed 2.8× the CPU steal of main's, and merkle uploads lean on it hardest.
    • A run with the code swapped between the two fleets' droplets would settle it.
  • An earlier, unmatched run is superseded. Its clients linked different saorsa-transport versions, and its downloaders ran closed-loop, so the faster fleet carried 1.85× the reads.
  • Full report: ant-testnet/state/comparisons/v2-1358-kad-vs-main-matched-60m-1009/README.md.

New dependency

none

ADR

ADR-0020: Kademlia-style chunk GET: return the chunk or closer peers, added in this PR.

Mitigation / rollback

The node is passive until clients send the request. To stop clients using it, drop the user-agent token; they then fall back to FIND_NODE plus GET. Otherwise revert.

🤖 Generated with Claude Code

[dirvine]

Code approval at the reviewed head

No introduced blocking runtime defect found in this PR. Production responses use the size-capped encoder; authenticated requester identity is passed into the handler. Decoder use here is test-only.

Stack coordination remains required: saorsa-core#173 has a P2 decoder-size blocker. Refresh the core patch pin at Cargo.toml:240 after that fix and rerun integration tests; propagate the resulting node head into the client. This code approval is not approval to ship the current linked dependency set or a waiver of merge/release gates.

Reviewed commit: 79385bfddc04a26b5c0747b3355364c12b9a127b.

Review method: six substantive seats across the linked set, including a completed z-ai/glm-5.2 API review recovered from a worker that timed out after saving its response; coordinator source checks and local execution. Initial functional/security/test seats recommended approval. The coordinator probe, independent adjudicator, API seat and replacement concurrency/release seats corroborate the missing decoder bound. This is not unanimous approval. The API seat recommended client approval with concerns; the coordinator withholds it because the exact client pin still exposes the defect.

Local verification (macOS, rustc 1.96.0): core closer-peers 3 passed and DHT-manager regression 143 passed; protocol all-features 108 passed; node get-or-closer unit tests 3 passed and real-transport E2E 1 passed; client kad_read unit tests 10 passed and real-transport E2E 1 passed. Formatting passed in all four repositories. No production/testnet deployment or merge performed. Draft git pins were verified with cargo metadata --locked; this is not published dependency closure. ADR-0020 remains Proposed.

[mickvandijke]

Thanks.

• ad88647 moves the saorsa-core pin to fd37775, the bounded closer-peers decoder. The 3 get-or-closer unit tests and the real-transport e2e pass, and CI was green (21/21).
• 900ff66 updates ADR-0020:
  • the 64 KiB bound on closer peers;
  • the client dropping a peer whose content fails verification (ant-client#221, 7d9a248);
  • that every queried peer can now send content, listed as a trade-off;
  • the validation list.

[dirvine]

Re-review — pin update confirmed, one coordination item

Fix verified

• saorsa-core pin now fd377754 (the bounded closer-peer decoder) in both Cargo.toml and Cargo.lock — the first-round coordination requirement is met.
• ADR-0020 docs updated to match: oversized closer-peer payloads rejected before parsing; bad-content peers dropped.

Coordination item (not a code defect)

• The PR head is 97759205, but ant-client#221 pins ant-node at ad886472 — a commit inside this PR's branch, not the head. The client consumes the get-or-closer handler and the fixed core pin from that mid-branch commit.
• Please either: (a) pin the head after squash/merge, or (b) confirm ad886472 is the intended node revision. Otherwise the client's node dependency tracks an unreferenced commit once this branch merges.

Local verification

• Handler + e2e tests at head 97759205: 3/3 and 1/1 ok (local run, macOS).

Verdict

APPROVE at head 97759205, subject to the node-revision pin being reconciled before the client ships. Panel: GLM 5.3 seat billing-failed (402) — 2/3 reviewed, provisional.