Repository navigation
AI Vision board: embeddedci.aivision client, pytest fixture, vision_* MCP tools - #62
Merged
Merged
Conversation
embeddedci.aivision.AIVision wraps /api/v1 of ai-vision-agent with the standard library: status, capabilities, lease (renewed in the background), camera, frames, zoom, snapshots, calibration, screen health, identify, detect, ocr, find, compare, waits (split into requests of at most 60 s), checks, references, models, actuators, press/tap/swipe, recordings and evidence. Error codes map to one exception each. mDNS discovery of _aivision._tcp behind the existing discovery extra. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The aivision fixture opens the board from --aivision / AIVISION_CONNECTION (skips without one), holds its lease for the session, and saves the screen into --aivision-artifacts when a test fails. Unit tests run against a fake board with the agent's response shapes; test_aivision_live runs the same calls against the agent's simulator or a board. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
13 tools on the same server, with their own connection (vision_connect, or AIVISION_CONNECTION on first use): status, look (the screen as an image content block, or a zoomed region), ocr, find, wait_for, compare, screen_health, calibrate, add_reference, press, tap. Errors carry the board's hint (the visible text for a press that finds nothing). README table, server instructions and the tool surface snapshot updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs/aivision.md (every call, result types, errors, pytest options, the simulator), a section and contents link in the SDK README, changelog entries for embeddedci and embeddedci-mcp. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The agent caps verify_timeout_ms on press, tap and swipe at 90 s (one request has to finish before Cloudflare cuts it). The SDK refuses a longer verify_timeout before sending (MAX_VERIFY_S), and vision_press accepts at most 90 s instead of 120. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the client side for the AI Vision board (ai-vision-2,
embeddedci-com/ai-vision-agent), on the LAN. No cloud path yet: that needs server work.SDK:
embeddedci.aivision(standard library only, no new dependency)AIVisionwraps the agent's v1 HTTP API: calibration, frames and zoom, OCR, find, detect, compare, identify, screen health, waits and checks, references, models, actuators (press,tap,swipe,press_key), recordings, evidence, and the board's lease (renewed in the background).NotCalibratedError,BusyError,WaitTimeout,NotFoundError, ...).discover()browses mDNS_aivision._tcpwith the existingdiscoveryextra.aivisionfixture,--aivision,--aivision-token,--aivision-lease,--aivision-artifacts. A failing test saves the screen it saw.MCP: 13
vision_*tools on the same server, with their own connection:vision_lookreturns the screen as an image content block (a first for this server), plus ocr, find, wait_for, compare, screen_health, calibrate, add_reference, press, tap, connect, disconnect, status. The tool surface snapshot only gains entries.Docs:
packages/embeddedci/docs/aivision.md, a section in the SDK README, the MCP README tool table, server instructions, changelogs (Unreleased).Tests
test_aivision.py) and against a stub client (test_vision_tools.py).test_aivision_live.py,test_vision_live), skipped withoutAIVISION_CONNECTION. Ran green against the agent's simulator atc8ba9ca(make sim).make test: all four packages pass.Release note:
embeddedci-mcpneeds itsembeddedcipin raised to the release that shipsembeddedci.aivision.🤖 Generated with Claude Code