Skip to content

test(cache): give each remote cache e2e case a long-lived backend - #807

Draft
wan9chi wants to merge 2 commits into
mainfrom
remote-cache-e2e-lifetime
Draft

wan9chi wants to merge 2 commits into
mainfrom
remote-cache-e2e-lifetime

Conversation

@wan9chi

@wan9chi wan9chi commented Oct 5, 2026

Copy link
Copy Markdown
Member

Motivation

The remote cache e2e cases should run against the public cache service from #718, the backend users actually deploy. That service is a long-lived deployment: its endpoint is also the audience of upload tokens, and it keeps its state for as long as it runs. remote-cache-server COMMAND started a fresh backend on a new port for every step, and the steps shared state through files.

This stack gets the test infrastructure in place first, against the existing file backend. After #718, one change then switches the backend without changing any snapshots.

Changes

  • One backend per case. Each case starts its backend in its first step and stops it in its last:
    • remote-cache-server start runs the backend in the background, in its own session and without the terminal's file descriptors. The step can then finish, and Ctrl-C in later steps doesn't reach the backend. It writes the endpoint to remote-cache/server.json.
    • remote-cache-server run COMMAND [ARGS...] runs the command against that endpoint, then prints the request lines it caused, as the wrapper did.
    • remote-cache-server stop stops the backend. It fails if the backend answered with a 5xx status.
    • A backend whose case directory is gone, or that has been idle for ten minutes, also stops. That covers a case that times out before its stop step.
  • Tap. The endpoint is a tap that forwards requests and responses unchanged and records the request lines, so backends no longer log themselves. Apart from the added steps and the run in each command line, the snapshots are unchanged.
  • Dropped remote_cache_backend. That fixture tested the file backend's own protocol through cbor-http, including state shared across wrapper invocations. The file backend is replaced later in this stack, and the service has its own protocol tests. The fixture, cbor-http, its EDN formatting and unit test, the cbor-edn catalog entry, and the CI steps that ran the test are removed.

The harness doesn't change. Cases manage their backend through their own steps.

🤖 Generated with Claude Code

wan9chi and others added 2 commits October 5, 2026 12:33
The `remote_cache_backend` fixture tested the Node test backend's own
protocol through `cbor-http`, including state shared across wrapper
invocations. The next change gives each e2e case one long-lived backend,
and the backend is later replaced by the public cache service, which has
its own protocol tests. The `remote_cache` cases keep exercising the
backend through `vp run`.

Remove the fixture, `cbor-http`, its EDN formatting and unit test, the
`cbor-edn` catalog entry, and the CI steps that ran that test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`remote-cache-server COMMAND` started a backend for one command, so a
case with several steps started several backends that shared state
through files, each on a new port. Each case now starts one backend in
its first step and stops it in its last:

- `remote-cache-server start` runs the backend in the background, in
  its own session and without the terminal, so the step can finish and
  Ctrl-C in later steps doesn't reach it. It writes the endpoint to
  `remote-cache/server.json`.
- `remote-cache-server run COMMAND` runs the command against that
  endpoint and then prints the request lines it caused.
- `remote-cache-server stop` stops the backend and fails if the backend
  answered with a 5xx status. A backend whose case directory is gone, or
  that has been idle for ten minutes, also stops.

The endpoint is now a tap that forwards requests and responses unchanged
and records the request lines, so the backend behind it no longer logs.
Apart from the added steps and the `run` in each command line, the
snapshots are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  -0.21%  [ -5.50% ..  +6.01%]  overhead  +304.45%
dynamic/access             change  -0.44%  [ -2.38% ..  +1.20%]  overhead   +14.26%
dynamic/access-relative    change  +0.10%  [ -2.29% ..  +1.57%]  overhead   +58.74%
dynamic/access-contended   change  +0.58%  [ -2.06% ..  +6.53%]  overhead   +16.64%
static/launch              change  +0.73%  [ -4.77% ..  +5.68%]  overhead  +738.19%
static/access              change  -0.27%  [ -2.86% ..  +2.16%]  overhead  +810.86%
static/access-relative     change  +0.33%  [ -1.47% ..  +2.34%]  overhead +1383.45%
static/access-contended    change  -0.21%  [ -3.63% ..  +2.19%]  overhead +3117.12%

macos

dynamic/launch             change  +0.03%  [ -4.06% ..  +5.03%]  overhead  +227.07%
dynamic/access             change  -1.02%  [-47.05% ..  +4.08%]  overhead    +5.38%
dynamic/access-relative    change  +0.57%  [ -6.93% ..  +7.48%]  overhead  +251.05%
dynamic/access-contended   change  +1.02%  [ -7.46% .. +13.12%]  overhead    +1.78%

windows

dynamic/launch             change  +2.34%  [ -9.71% .. +15.26%]  overhead   +27.69%
dynamic/access             change  +0.18%  [-11.51% .. +10.36%]  overhead    +2.31%
dynamic/access-relative    change  +0.35%  [ -2.97% ..  +2.65%]  overhead    +1.10%
dynamic/access-contended   change  -0.71%  [-16.50% .. +15.19%]  overhead    +5.74%

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant