Skip to content

feat(webrtc): raise browser listener limits, and ADR-0023 for hot reads and full-session refusal - #250

Open
mickvandijke wants to merge 5 commits into
mainfrom
mickvandijke/v2-1508-adr-0023-hot-reads
Open

mickvandijke wants to merge 5 commits into
mainfrom
mickvandijke/v2-1508-adr-0023-hot-reads

Conversation

@mickvandijke

@mickvandijke mickvandijke commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

ADR-0023 covers what happens when thousands of browsers want the same chunks, for example everyone watching the same film, and what we change first. This PR also raises the browser listener's session limits.

Findings it records

  • Every client reads a record from its nearest holder first. A hot record is served by one node while the other six in its close group sit idle.
  • A node at its WebRTC Direct session limit ignores new browsers. The browser waits out a 10 s timeout, then blocks that node for 30 minutes. Busy nodes therefore disappear from clients' view exactly when demand peaks.
  • For synchronised demand, upload bandwidth runs out before session slots do. 1,000 viewers of a 5 Mbit/s stream ask for about 5 Gbit/s of one chunk.

Decision

  1. Clients spread first reads uniformly across the close group, then fall back to nearest-first.
  2. A transport with no free slot answers a proven browser with a STUN error (486) instead of silence. This amends ADR-0015's rule of rejecting without an error response, for transport capacity refusals.
  3. Browsers treat a dial the node declined as Refused, with a 15 s cooldown instead of 30 minutes.
  4. This PR's code change: the listener defaults go from 32 to 256 sessions (the transport's ceiling), 32 to 512 channels, and 16 to 512 concurrent requests.
    • Channels and request slots are sized at two per session. On the testnet, browsers held 1.4 channels per session at the median and 1.9 at p90, so channel slots ran out before session slots. The old 32 channels filled at 18–23 sessions. At 192 browsers, 256 channels and 128 request slots both filled at 194 sessions.
    • The per-IP quota, the rate buckets and the in-flight byte budgets are unchanged, so frame and response memory stays bounded as before.

Each of the following needs its own ADR, and all are deferred:

  • raising the transport's 256 ceiling;
  • caching hot records on more nodes;
  • batched or delegated lookups.

Implementation PRs

Linear issue

Closes V2-1508

Risk tier

  • T0 — docs / tooling / CI / pure UX-output. Repo CI only.
  • T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
  • T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
  • T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Compatibility

  • Wire: none
  • Storage: none
  • API: none
  • Config: three default values change. A node whose --config file sets them keeps its values. The new defaults satisfy every startup invariant: per-IP sessions × channels per session = 8, which is below both 512 channels and 512 requests.

Semver impact

  • breaking
  • feature
  • fix

Test evidence

  • cargo test --lib -- web_rtc config: 108 passed. This includes the test that checks the default config satisfies the listener's invariants.
  • cargo clippy --all-features --lib -- -D clippy::panic -D clippy::unwrap_used -D clippy::expect_used: clean. cargo fmt --check: clean.
  • python3 scripts/adr-governance.py: passed (19 ADR files).
  • Local real-browser run (7-node devnet, limits 2 sessions / 1 per IP): a full node turned the second browser away in 67 ms in Chromium and 54 ms in WebKit, against 10,011 ms and 10,005 ms on the unpatched transport. After the 15 s cooldown the browser connected again, where the baseline still blocked it.
  • Dev testnet A/B (2026-10-13), recorded in the ADR's Validation section:
    • Setup: 60 nodes, 18 of them behind NAT. 64 headless-Chromium browsers, each on its own IP, all download the same 50 MiB file at once, over 18 rounds.
    • Today's limits: 61–66% of downloads completed, with today's client and with the new one alike. Every failure was a false "record not found".
    • New limits: 384 of 384 completed. Nodes peaked at 203 MiB and 1.2 cores, against 184 MiB and 1.25 cores before.
    • Scale test, 192 browsers at once:
      • 256 channels and 128 request slots: 762 of 768 completed, with about 2,300 channels closed per round.
      • 512 and 512: 768 of 768 completed, with no channel or request rejections, at 268 MiB and 1.3 cores peak per node.
    • Memory cost: about 0.5 MiB per session.
    • Follow-up found, ant-client: a read whose holders are all refusing reports "record not found" before their 15 s cooldown ends. With the new limits that only happened once channel or request slots filled.

New dependency

none

ADR

This PR adds ADR-0023 (Proposed).

Mitigation / rollback

Revert the commits, or have operators set max_connections, max_channels and max_concurrent_requests in [webrtc_direct] of their --config file to the old values.

🤖 Generated with Claude Code

…essions

Records why every client read a popular record from one holder, why a full
node looked unreachable for 30 minutes, and the decision to spread first reads
across the close group and answer capacity refusals with a STUN error. Amends
ADR-0015's capacity-rejection rule and includes Chromium/WebKit measurements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mickvandijke and others added 2 commits October 12, 2026 16:10
- A transport refusal requires that ICE never connected; DTLS failures,
  authentication failures and client-initiated closes keep the 30-minute
  cooldown, and a full failure cache evicts the soonest-expiring record.
- Record two trade-offs: a browser replacing its own association can hit its
  source quota, and only the admitting request is refused.
- Update the real-browser measurements to the final client build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Spreading applies to chunk reads (pointer reads already ask the whole close
group); known-peer fallbacks keep nearest-first; shutdown attempts stay
silent; precise eviction and ICE-state wording; application-level closes from
old nodes also get 15 s; admission tests listed under their module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mickvandijke
mickvandijke marked this pull request as ready for review October 12, 2026 16:18
Browser sessions go from 32 to 256, the transport's ceiling. Channels follow
sessions (32 to 256), and request slots keep their one-per-two-sessions
ratio (16 to 128): the PQ handshake and every request take a request slot,
so more sessions without more slots would close their channels instead.

The per-IP quota, rate buckets and in-flight byte budgets are unchanged, so
frame and response memory stays bounded as before. ADR-0023 records the
change and asks the network A/B to measure the per-session cost.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mickvandijke mickvandijke changed the title docs(adr): ADR-0023 spread hot-record reads and refuse full browser sessions feat(webrtc): raise browser listener limits, and ADR-0023 for hot reads and full-session refusal Oct 13, 2026
The testnet A/B showed browsers hold 1.4 channels per session at the median
and 1.9 at p90, so channel slots ran out before session slots: the old 32
channels filled at 18-23 sessions, and with 256 sessions, 256 channels and
128 request slots both filled at 194 sessions under 192 browsers, closing
about 2,300 channels a round. At 512 channels and 512 request slots no
channel or request was rejected and all 768 downloads completed, at the same
node memory (268 vs 271 MiB peak).

ADR-0023 records the network A/B: at 64 synchronised browsers, today's
limits completed 61-66% of downloads, all failures being false "record not
found", and the new limits completed all of them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant