Repository navigation
computer: updates to make workspace easier to extend - #214
Conversation
🦋 Changeset detectedLatest commit: 521091f The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
| export function globOne(value: string | readonly string[] | undefined): string | undefined { | ||
| const list = globList(value); | ||
| if (list === undefined) return undefined; | ||
| return list.length === 1 ? list[0] : `{${list.join(",")}}`; |
There was a problem hiding this comment.
🟡 Comma-bearing include globs lose matches
When grep receives multiple includes with a literal comma, globOne splits that filename into separate brace alternatives. Matching files disappear from search results.
Learn more
The grep tool converts multiple include globs to one pattern for the filesystem search. The brace parser treats every top-level comma inside that pattern as an alternative separator, including commas originally inside a filename. The resulting pattern no longer matches the filename, and grep silently omits its matches.
Example: With include: ["src/a,b.ts", "src/c.ts"], the generated pattern is {src/a,b.ts,src/c.ts}. The src/a,b.ts file does not match any of those alternatives.
Recommended fix: Preserve each input pattern as an independent alternative without interpreting its literal commas as separators. This may require adding a list-of-includes path to the find/grep walker or implementing an escaping convention in the glob compiler and encoder together.
Was this helpful? React with 👍 or 👎 to provide feedback.
| const resolve = async (input: unknown): Promise<string> => { | ||
| if (typeof input !== "string") throw new TypeError("path must be a string"); | ||
| const path = normalizeRootedPath(base, input); | ||
| if (typeof fs.lstat !== "function") return path; |
commit: |
02345f1 to
80c7e59
Compare
80c7e59 to
b95930a
Compare
b95930a to
ed71643
Compare
a2ca0a6 to
92c6b63
Compare
92c6b63 to
aff3427
Compare
aff3427 to
90b0ff9
Compare
A host module factory received the Workspace's Git client, Artifacts client and runtime, but not its assets client, so a module that publishes files had to close over a client it was given separately. The Workspace now passes its assets client to a module backend when it connects, and the JavaScript backend hands it to each factory. It is absent when the Workspace has no assets configured.
Isolate code could reach Git, Artifacts and the container through host modules, but had no way to publish a file it produced. ws:assets exports publish(path, options), which uploads a workspace file through the Workspace's assets client and returns a time-limited URL. The expiry is milliseconds or a duration such as "30m", with a one hour default that the module's options can change. The filename, disposition and content type pass through to the client. The path resolves against the backend root like any other host call, and the module fails when the backend connects if the Workspace has no assets client.
ws:artifacts could create, list and delete repositories but not mint
the git token that cloning or pushing one needs, so isolate code had
to fall back to the shell's artifacts command.
createToken(name, scope?, ttl?) mints a token and returns its id,
plaintext, scope and expiry. share(name, { scope?, ttl? }) returns the
repository's remote with a token embedded, the same URL that
`artifacts share` prints, and resolves the repository before minting
so a missing one fails without leaving a token behind. Both default to
a read token, take the ttl in seconds or as a duration, and refuse a
write token on a read-only backend.
A deployment often wants every container command to start from the
same shell state: pipefail, so a failing stage of a pipeline is not
hidden by the one after it, or exported variables such as a git
identity. ws:container passed the command through as written, so the
only way to get that was to wrap the module and rewrite each command.
createContainerModule({ prelude }) now runs the prelude before each
command, joined by a newline. A newline keeps a command that opens
with a comment or a shebang intact, which a semicolon would not, and
does not skip the command when the prelude's last status is non-zero,
which && would. The command is still checked for emptiness before the
prelude is added.
ws:container capped each stream by bytes, and the runtime then kept
up to its own 2000-line default. A deployment that shows the model a
few hundred lines elsewhere could not ask for the same here.
createContainerModule({ maxOutputLines }) now passes a line limit to
the runtime alongside the byte limit. Leaving it unset keeps the
runtime's default.
WorkerJavaScriptBackend built its description once, in its constructor, from each module's description at that moment. A module whose description depends on something that exists only later, such as a module listing the agent's tools, which are built after the Workspace the backend belongs to, could only describe itself in general terms. The backend now reads each host module factory's description whenever its own description is read, so a factory can define description as a getter and the exec tool, built afterwards, sees the current text.
The exec tool cuts each stream to a number of lines and bytes, and
passes the same limits to the runtime so that whatever the model does
not see is saved to a file. createAITools, createPiTools and
createTanStackTools could set those limits only through the deprecated
`shell` option, because `exec` names backends and nothing else, so a
caller had to choose between the current option and its own limits.
`execOutput: { maxBytes?, maxLines? }` now sets them alongside
`exec`, and wins over limits given through `shell`.
createPiTools turned a tool's structured output into JSON text for the model and dropped the value itself. pi carries a tool result's `details` beside its content for the caller and its interface, so a caller that wanted the exit code or the stdout of an exec call had to parse the model's text back. The dispatcher now returns the tool's output as `details`. A read that returns an image leaves it off, since the output holds the encoded bytes already sent as content, and a call that fails validation or throws has none.
Each tool set builds the exec tool from defineExec: a description, an input schema, and an executor that yields snapshots. It was not exported, so a caller registering tools with a library the package has no adapter for had to go through createPiTools or the AI SDK's createExecTool, and route calls through that library's shape. @cloudflare/computer/tools/core exports defineExec and its types. It depends on zod and nothing else, so importing it does not pull in an agent library.
exec declared `input` as any JSON value. Offered a field that may be a string, Anthropic models sent the object they meant as a string of JSON, which the module then received as text. Checked live, Claude did this on every call, whatever the field's description said. With the field declared as an object and described as an "input object", Claude, GPT and the Workers AI default each sent an object on every call. `input` is now an object of JSON values, and the descriptions call it an input object. A top-level array, string or number is rejected, and a null under strict sampling still means the field was left out.
Declaring `input` as an object is what stops models sending it as a string of JSON, but nothing makes every model follow the schema, and a string now fails validation and costs the model a turn. The exec definition gains prepareArguments, which replaces a string `input` that parses to a JSON object with that object, and leaves anything else as sent, so it fails validation with the schema's own message. It never mutates its argument. createPiTools runs it before validating an exec call. It does nothing when no backend is callable, since `input` is not offered then.
An exclusion such as "node_modules/**" matched everything below node_modules but not node_modules itself, so find returned the directory and the walker still listed its children before dropping each one. The tools describe exactly this pattern as skipping a directory along with everything below it, and callers had to pass both "node_modules" and "node_modules/**" to get that. An exclusion ending in /** now also excludes the directory it names, so the walk prunes it without listing it. A file of that name is kept, since the pattern only ever named what is inside a directory. The same walk serves recursive grep.
The glob matcher behind find, recursive grep and their exclusions
supported *, ** and ?, and treated a brace as literal text. A pattern
such as "**/*.{ts,tsx}", which models and people write by habit,
matched nothing, and the result was indistinguishable from no files of
that kind existing.
A brace group with at least one top-level comma now matches any of its
alternatives, and groups nest. A brace with no comma, or with no
closing brace, is still literal, as it is in a shell.
The grep tool took `include` as one glob and `exclude` as a list, and the find tool took `exclude` as a list, so a model had to remember which field was which shape and lost a turn to validation when it guessed wrong. Each of these fields now accepts a string or a list of strings. Several include globs are joined into one brace group, which the glob matcher reads as alternatives, and blank globs are dropped.
The edit tool diffed the whole file after every edit and returned the diff and patch whole. Diffing costs roughly the square of the number of differing lines: one changed line in a large file is cheap, but rewriting a few thousand lines takes seconds of CPU, inside a Durable Object holding the session, for a diff nobody reads. The output had no bound either, so a run of large edits could fill the transcript. The edit still applies in full. When the replaced text, counted in lines on either side of each edit, exceeds maxDiffLines (2000 by default), the diff is skipped. Otherwise the diff and patch are each cut to maxDiffBytes (128 KiB by default) on a character boundary. Either way the result sets diffTruncated.
The file tools passed whatever path the model sent straight to the Workspace, so a tool set meant for /workspace could read, write or delete anywhere in the filesystem, including through a symbolic link under /workspace that points elsewhere. Callers fenced paths by wrapping every tool. `root` on the tool set options now confines read, write, edit, delete, ls, find, grep and publish to one directory. A relative path resolves against the root, `.` and `..` are resolved before the check, and a path that leaves the root is refused. When the filesystem has lstat, a path with a symbolic link anywhere below the root is refused too. Each refusal comes back as the tool's ordinary error result, and nothing is touched.
The isolate JavaScript guide gains a section for ws:assets, and covers ws:artifacts' createToken and share, ws:container's prelude and maxOutputLines, the assets client on a module factory's host, and the lazily read module description. Both entry point tables list the new modules and @cloudflare/computer/tools/core.
The filesystem interface now describes brace alternatives in find and grep globs, and that an exclusion ending in /** also prunes the directory it names. The examples drop the paired "node_modules" and "node_modules/**" exclusions that are no longer needed, and the comment above the glob compiler lists brace groups.
The tool interface now covers the root option and how it refuses paths, execOutput, the details pi results carry, exec's input object and the repair of one sent as text, defineExec on @cloudflare/computer/tools/core, one or several globs for grep and find, brace globs, and the edit tool's diff bounds.
One changeset per change, each a line with a link to its documentation. Declaring exec's input as an object and pruning the directory a trailing /** exclusion names are minor, since each changes an existing behavior; the rest are additions and are patches.
createToolBindings described ws:tools without naming a single tool, because the backend used to read module descriptions once, when it was constructed, before a function of tools could run. The backend now reads them each time it describes itself, so the module can list the names code can import, which is what a model needs to call them without guessing. The description lists each export under its normalized name, leaving out excluded tools. A function of tools is called again each time, so the list is current when the exec tool is built. A list is read as given, but any other iterable may be single-use and is left for the connection, and a function that throws leaves the list out rather than failing the description.
…ited A ws:tools call could be cancelled after the isolate queued it but before the host dispatched it, and the binding still ran. A tool that does not watch context.signal would then go on to change the Workspace after the execution was cancelled. Check the signal just before calling the binding, as ws:assets and ws:container already do, so a cancelled call rejects with the abort reason instead of starting the tool. A tool already running when the signal aborts is still responsible for honoring it.
publish uploaded a workspace file and returned a public link without looking at the backend's access, so isolate code on a read-only backend could still hand files out. The read-only tool set already leaves out the publish tool, and ws:container, ws:git, ws:artifacts and ws:tools all refuse their writes there. Check context.access before anything else, and throw the same "requires Workspace write access" error the other modules do.
A Workspace with `output: false` saves nothing and returns each stream whole. ws:container then cut it itself, but only to maxOutputBytes and from the front, so maxOutputLines had no effect there and the result kept the start of the output while the module tells the model it keeps the last lines. Run the stream through the runtime's own OutputWindow with the same byte and line limits, the line limit defaulting to the runtime's 2000, and put a note in front saying which lines are shown. There is no file holding the rest, so the note is all the isolate gets in place of `truncated`.
886b2ab to
714af3f
Compare
The vitest require-to-throw-message rule now runs on test files, and three assertions added alongside ws:assets and the file tools' root checked only that a call rejected. Match the message each one expects: "no such path" for a file the confined tools must not have written, and "aborted" for a publish cancelled before it starts.
This PR adds functionality external harnesses need to either add themselves or work around in order to extend the defaults.
Isolate JavaScript modules (what code in the JavaScript backend can import)
ws:assets(new): publish a workspace file and get back a time-limited public URL.ws:artifacts: can now mint git tokens (createToken) and credentialed clone URLs (share), so code can clone and push Artifacts repos.ws:container:preludeoption runs fixed shell setup before every command, such asset -o pipefailor a git identity;maxOutputLinesoption caps output by lines as well as bytes.host.assets).ws:toolslist tools that are created later.The
exectoolinputis now an object, not any JSON value. Models were sending it as a string of JSON, and this stops that. If one still arrives as a string, it's parsed back into an object.execOutput, instead of only through the deprecatedshelloption.defineExecis exported from a new@cloudflare/computer/tools/coreentry point, so any agent library can build the exec tool without loading AI SDK or pi.pi tools
createPiToolsresults now include each tool's raw output asdetails, such as exec's exit code and output, so callers don't have to parse the text sent to the model.File tools
rootoption: keeps every file tool inside one directory. Relative paths resolve against it, and paths outside it or through symbolic links are refused.includeandexclude, and find'sexclude, accept one glob or a list.diffTruncatedflag. Large rewrites used to cost seconds of CPU and flood the transcript.Glob matching (in
dofs, which also changes what the find and grep tools match)**/*.{ts,tsx}matches both. Before, braces were treated as literal text, so such patterns silently matched nothing.node_modules/**now also skips thenode_modulesdirectory itself, so you no longer have to list both forms.