Conversation
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
fspy benchmarklinuxmacoswindows |
Remote cache stagingCommit: Cloudflare staging deployment and smoke tests passed. You can now perform manual verification. Endpoint: https://vp-cache-ci-staging.voidzero-docs.workers.dev/projects/manual Download the remote-cache-e2e-37270179886-1 artifact from the workflow run. It contains PR checks cover public reads and rejected writes. Main-branch push checks also cover authorized HTTP stores. This staging endpoint is shared by all PRs and main. A later deployment replaces its code and manual fixture. Compare the deployment ID in the response with the artifact before manual verification. Closing this PR does not remove staging. |
194d39f to
e461d47
Compare
|
Deploy prompt:
|
df2d129 to
e127e9b
Compare
e127e9b to
d4b0fea
Compare
58347c9 to
c2f00bd
Compare
## Motivation The public cache service (#718) follows its RFC and answers a fetch that matches neither key with HTTP 404 and a plain-text body. It never sends `kind: "not_found"`. The client and the Node test backend still used a 200 response with `kind: "not_found"`, so a miss meant different things depending on the server. This switches both to 404 and drops the `not_found` kind, giving the client and both servers one miss contract. ## Changes - `vt_remote_cache`: `Client::fetch` returns `Result<Option<Fetched>, Error>`, with `None` for a 404 response. `Fetched` only describes the body of a 200 response, so its `NotFound` variant is removed. A 200 response with `kind: "not_found"` is now a malformed response. Every other non-200 status is still an error, and a 404 download still fails. - Test backend (`packages/tools`): a fetch miss gets a 404 with the body `Not found`, logged as `POST /fetch 404`. - The remote cache e2e snapshots change only in those backend lines and responses. The `vp run` output is unchanged. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
c2f00bd to
0c55376
Compare
This reverts a463bde. The stall cases test how vp run handles a backend that never answers, not the backend itself, and the real cache service from #718 will replace remote-cache-server in these tests. That service can't stall, so --stall would have to move into its proxy, and the cases would stay skipped on Windows and musl. vtt stalled-remote-cache --fetch-miss runs on every platform without Node.js. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…795) ## Motivation The e2e cases for a remote cache that never answers need some requests to stall while others behave normally. For example, an upload case needs its fetch to reach a backend and miss before the upload stalls. `vtt stalled-remote-cache` was its own endpoint and stalled every request, so it couldn't do that. #718 will replace `remote-cache-server` with the real cache service, which can't stall requests itself. So the stalling has to happen in front of whatever backend a test runs. ## Changes - **Proxy.** `vtt stalled-remote-cache [--stall ROUTE]... COMMAND [ARGS...]` is now a proxy for the endpoint in `VP_REMOTE_CACHE_URL`, which must be `http://<host>:<port>/<path>`. It runs the command with `VP_REMOTE_CACHE_URL` set to the proxy. - A request to a stalled route below the endpoint, such as `/store`, is never answered, and it emits a `stalled` milestone. - Other requests are forwarded with `connection: close`, so each one comes in on its own connection and is checked on its own. - A forwarded request gets a 502 if the endpoint can't be reached. - **Fetch cases.** `ctrl_c_during_fetch` and `fast_fail_during_fetch` stall `/fetch`, with an unreachable endpoint behind the proxy. They still run on every platform without Node.js. - **New `ctrl_c_during_upload` case.** It runs `remote-cache-server vtt stalled-remote-cache --stall /store vt run build`. The fetch reaches the backend and misses, the upload stalls, and Ctrl-C cancels it. - **`remote-cache-server`** now ignores Ctrl-C and leaves it to its command, so it still prints its request log afterwards. ## Notes for reviewers - **Lost milestones on Windows.** A milestone there is the console title, and ConPTY sends it on its next render. If the command sets a milestone at about the same time as the proxy, the earlier one can be lost. The doc comment says so. Cases that need `remote-cache-server` are skipped on Windows anyway. - **Conflict with #718.** #718 rewrites `packages/tools/src/remote-cache/cli.ts`, so the line that ignores Ctrl-C needs to move into its version. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
0c55376 to
c4d5b0b
Compare
## Motivation The self-hosted remote cache server in #718, designed in #716, serves reads to anyone but accepts a store only with a GitHub Actions OIDC token. Other servers will need other credentials. For example, a private cache behind Cloudflare Access could need headers on every request. This adds one general hook so each kind of credentials is a separate implementation, and the client doesn't need to know about any of them. ## Changes - `vt_remote_cache::auth::Auth` supplies the headers for each request, given its operation: fetch, download, or store. It can use the client's HTTP client to get credentials, such as a token, and that client doesn't follow redirects. If it fails, the request isn't sent, and the operation fails with the new `Error::Auth` ("failed to authenticate"). - `Client::new(endpoint, auth)` takes the auth. `Anonymous` adds no headers. - Planning resolves how requests authenticate into `remote_cache.auth`, next to the access mode and endpoint. The result holds everything needed to build the credentials, so nothing reads envs after planning. Choosing the auth from `cache.remote` config or envs later only changes this step. For now, the only kind is `anonymous`. - `vt` turns the resolved auth into a `vt_remote_cache` auth with `build_auth`, a single `match`, and caches clients by endpoint and auth. Requests don't change. Plan snapshots gain `"auth": {"kind": "anonymous"}`. The next PR in this stack adds GitHub Actions OIDC as the first auth with credentials. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…DC (#798) ## Motivation The self-hosted remote cache server in #718, designed in #716, accepts uploads only with a GitHub Actions OIDC token. The token's audience must be the namespace endpoint, and it must come from a push job on the main branch. `vp run` sends stores without credentials today, so that server rejects every upload with 401. ## Changes - Planning resolves `remote_cache.auth` to `github-oidc` when `ACTIONS_ID_TOKEN_REQUEST_URL` and `ACTIONS_ID_TOKEN_REQUEST_TOKEN` are set in the envs visible at the `vp run` level. That happens in jobs with `permissions: id-token: write`; otherwise the auth stays `anonymous`. - It holds the request URL, the request token, and the audience, which is the endpoint without a trailing slash. - The request token is a `Secret`, which debug output and serialized plans redact. - `build_auth` turns `github-oidc` into `vt_remote_cache::auth::GithubOidc`, which adds `Authorization: Bearer <token>` to stores only. Fetches and downloads stay anonymous. - It requests a token when the first store needs one. - Later stores reuse the token until two minutes before its `exp`. Cloudflare receives a store's whole body before the Worker checks the token, so the token has to outlast the upload. A token without `exp` is a malformed response. - Concurrent stores wait for the same request. - A failed request is remembered, so later stores fail right away without making more requests. Each task with a failed upload shows the existing "Not uploaded to the remote cache" warning. - Neither token appears in debug output or errors. - Its state is a single enum: ready with a request and an optional cached token, or failed. A token can't stay cached after a failure. - Tasks still receive the two env vars as untracked envs, as in #691, so npm trusted publishing through `vp run` keeps working. Stacked on #797, which adds the `Auth` hook and the resolved auth config. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
c4d5b0b to
172ad91
Compare
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Remove the obsolete cache size study and its RFC link. Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
Co-authored-by: GPT-6 Codex <codex@openai.com>
`npm_execpath` points to pnpm's standalone executable on the Linux and Windows runners, so running it through `node` failed. Run it directly unless it's a JavaScript entry point. The remote cache package also brought esbuild and workerd into the workspace, and a root `pnpm install` without `--ignore-scripts` failed on their unapproved build scripts. Allow them, as the standalone package already does. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
172ad91 to
b30dcd5
Compare
Add
packages/remote-cachefor the server in #716, using Cloudflare Workers, primary D1 metadata, and a private R2 bucket. The server supports public reads and GitHub Actions OpenID Connect checks for writes. Stores use streaming uploads, atomic publication, storage limits, and automatic data expiry.One workflow deploys related changes to a persistent staging Worker, D1 database, and R2 bucket. Internal PRs,
mainpushes, and manual runs share these resources. The workflow runs deployments and smoke tests in sequence, then updates PR comments with the tested revision and manual instructions. Each deployment replaces the previous revision. Closing a PR keeps staging available. Setup uses repository secrets and variables. Repository maintainers can configure staging without a GitHub environment.The e2e plan defines automated checks and manual exercises. PR runs check public reads and rejected writes. Pushes to
mainalso check authorized uploads, multipart storage, replacement, concurrency, and quotas. Local tests repeat smoke checks against reused storage and retain maximum-payload and scheduled-handler coverage. Real Cron execution and maximum payloads on Cloudflare require separate release exercises./fetchreturns only{ kind: "fallback", key }for fallback matches and does not read R2. Exact matches return the value andblob_id. A missing or unreadable exact value returns503. Fetch misses and unavailable blobs return plain-text404, as specified in the local RFC.Setup rejects incompatible origins and repositories before it changes existing policies or saved configuration. Each Worker has separate rate-limit counters that remain stable across revisions. Workers Free CPU support remains unverified. The guide recommends Workers Paid for the full payload limits.
Motivation
Maintainers need a cache service in their own Cloudflare account. Developers and fork contributors must reuse public task results without login. Only trusted jobs on
mainshould publish those results. Reviewers need one persistent staging environment and a clear verification result after each related change.