Repository navigation
feat(transcript): show Host image files referenced in assistant replies - #174
Merged
akemmanuel merged 2 commits intoOct 23, 2026
Merged
Conversation
…hem via the sidecar Desktop transcripts showed broken image tiles for uploaded screenshots while hosted Web worked. Two Desktop-only causes: - Transcript previews used the stored workspace URL (127.0.0.1:4096), but the Desktop Local Host runs as a sidecar on a per-launch port. - Unrestricted uploads went to the OS temp dir, which is outside the default allowed roots (home) and is wiped on reboot, so /api/fs/file rejected them. Uploads with a Project directory now always land in <project>/.opengui-uploads (self-ignored by git), matching restricted Hosts. Transcript previews resolve the sidecar URL for Desktop Local workspaces. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Assistant replies like `` rendered only their alt text: markdown images were blocked wholesale to avoid tracking pixels. Host files are not a third-party origin, so render them through the existing /api/fs/file route (path grants and allowed roots still apply) and keep remote URLs blocked. - Markdown images: Project-relative, absolute, and file: paths plus inline raster data URLs render with a lightbox; http(s) and active URLs stay alt text. A custom urlTransform keeps Windows drive paths that react-markdown would otherwise drop as an unknown protocol, and percent-encoding (spaces, umlauts, backslashes) is decoded before resolving. - Windows: /api/fs/file uses the platform's isAbsolute, so `C:\...` and UNC paths are no longer joined onto the Project directory; the Frontend treats UNC paths as absolute too. - Remote Hosts with a bearer token: `<img>` cannot send Authorization, so HostImage fetches with the token and shows a blob URL. Desktop Local and cookie sessions keep plain requests. Applies to uploads, file parts, tool images, and markdown images; failures show "Image unavailable". - Tool and file-part previews use the Desktop sidecar URL (attachmentBaseUrl) instead of the stored workspace URL; gif/webp tool images are allowed. - /api/fs/file sends nosniff, a sandboxing CSP (agent-written SVG cannot run script on the Host origin), and no-cache so regenerated images refresh. - System prompt tells models to reference images as . Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When the agent creates an image and replies with
, the transcript shows only the alt text.MarkdownRendererblocks every markdown image so that tracking pixels can't load. A file on the Host is not a third-party origin, so it doesn't need that block.Change
Markdown images in assistant replies
file:URLs, and inline rasterdata:image/*render as a preview. Clicking a preview opens the existing lightbox.http(s),javascript:, SVG data URLs, and files without an image extension still render as alt text./api/fs/fileroute, so path grants,OPENGUI_ALLOWED_ROOTS, andrealpathchecks apply unchanged.MarkdownImageContext(any otherMarkdownRendereruse), behaviour is unchanged.Paths and permissions across Shells and Hosts
/api/fs/fileuses the platform'sisAbsolute, soC:\…and\\server\shareare no longer joined onto the Project directory. The Frontend also treats UNC paths as absolute. A customurlTransformkeepsC:\…andfile:image sources, which react-markdown's default would drop as unknown protocols. Percent-encoding (spaces, umlauts,%5C) is decoded before the path is resolved.realpath(/var→/private/var) is already enforced by the Host. Nothing new needed.attachmentBaseUrl).ToolCallPartViewandFilePartViewstill used the stored workspace URL and requested the wrong port.<img>can't sendAuthorization. The newHostImagecomponent fetches with the bearer token and shows a blob URL. The Desktop Local credential is never sent to another Host. This also fixes uploaded-image thumbnails on token-authenticated Remote Hosts.<img>requests, as before.Hardening
/api/fs/fileX-Content-Type-Options: nosniff.Content-Security-Policy: sandbox; default-src 'none'; …: SVG written by the agent can't run script on the Host origin when opened directly.Cache-Control: no-cache: images the agent overwrites in place refresh in the transcript.Other
.attachments.openImageandattachments.imageUnavailablein en, de, and es.Not in this PR (follow-up)
readtool are stored under the Host data directory (tool-output/), and the tool-result projection doesn't carry them intostate.attachmentsyet. Restricted Hosts don't grant that directory, so the follow-up needs a session-scoped route (/api/host/sessions/:id/tool-output/:file, checked viareadSession) plus the projection fix.writeof*.png/*.svg.Testing
vp check: cleanvp test: frontend 541, backend + harness 706, all passingpnpm run slop-check: okclassifyMarkdownImageSource: POSIX, Windows, UNC,file:, percent-encoding, rejectionsresolveAttachmentAuthTokenMarkdownRenderer: Host route URL, Windows path, remote still blocked, placeholder, no-context fallbackHostImage: direct load vs. bearer fetch vs. 403 placeholder/api/fs/file: security headers and Project-relative path🤖 Generated with Claude Code