Skip to content

feat: ChatGPT web search and image generation for Codex models - #176

Merged
akemmanuel merged 1 commit into
masterfrom
feat/codex-web-search-image-gen
Oct 24, 2026
Merged

akemmanuel merged 1 commit into
masterfrom
feat/codex-web-search-image-gen

Conversation

@akemmanuel

Copy link
Copy Markdown
Owner

Summary

Runs that use a Codex model (ChatGPT subscription) now get two extra tools, both backed by the same ChatGPT endpoints the Codex CLI uses:

  • web_search calls backend-api/codex/alpha/search and supports search_query, open and find. Search state is scoped per Session, so later open/find calls can reuse ref ids from earlier searches. Citation markers are normalized to [turn0search0]. Results include up to 10 deduplicated http(s) sources, shown as links in the transcript.
  • generate_image calls images/generations, or images/edits when reference images are given (gpt-image-2). It supports transparent backgrounds and up to 5 reference images from the Project. Reference paths go through the Host path authorizer. Images are saved to <project>/.opengui-images/<toolCallId>.png; the folder gets its own .gitignore (*) and existing files are never overwritten. The model also gets the image as an attachment, and the transcript shows it pinned below the tool row.

Host policy (Settings → Integrations)

  • Web search: live, cached or off. Default: live.
  • Image generation: on or off. Default: on.
  • Shared offerings may use the Host's ChatGPT subscription. Default: off. Hidden on local single-user Hosts.

The policy is stored in Host settings and exposed at GET/PUT /api/host/codex-tools; PUT requires a human admin.

Harness changes

  • combineAgentToolSources merges several AgentToolSources (MCP and ChatGPT tools) and rejects duplicate tool names.
  • AgentToolScope now includes the run's model, and invoke receives toolCallId.
  • ModelToolDefinition.maxResultBytes lets a tool raise its result cap, clamped to 64KB. Search results use 32KB.

Errors

  • After a 401, the credential is refreshed once and the call retried.
  • 401, 403, 429 (usage limit), 5xx, timeout and network failures produce clear, redacted messages.

Testing

  • vp check is clean.
  • New unit tests for the ChatGPT tool source (availability, policy, shared offerings, request shape, 401 refresh, 429, citation normalization, image save and no overwrite, edits endpoint, denied reference path).
  • New Host integration tests (a Codex Session gets the tools, the search call uses the Codex credential, the policy survives a restart).
  • Updated route auth and validation contracts, harness agent-tool tests, and transcript and tool-call model tests.
  • Checked manually against the live ChatGPT backend: a search plus a follow-up open, and a transparent image generated and saved.
  • pnpm run slop-check passes.
  • The full vp test suite was not run here, only the affected areas (backend codex-tools, host, routes, harness agent tools, transcript, message-list).

🤖 Generated with Claude Code

Codex-model runs now get `web_search` and `generate_image` tools backed by
the connected ChatGPT subscription (the same backend endpoints the Codex CLI
uses). Generated images are saved to a gitignored `.opengui-images/` folder
in the Project and shown inline in the transcript; search sources are shown
as links.

A Host policy in Settings → Integrations controls web search (live, cached,
off), image generation, and whether shared model offerings may use the Host
subscription (off by default).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@akemmanuel
akemmanuel merged commit 513ffcc into master Oct 24, 2026
2 checks passed
@akemmanuel akemmanuel mentioned this pull request Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant