fix(tools): coerce string-typed integer tool arguments (Android tap tool error) - #317
Merged
Merged
Conversation
…types
`ToolCollection` passed the model's tool arguments straight into the tool
implementation without looking at the tool's `input_schema`. When a model
emitted integers as JSON strings (observed with Claude via Bedrock, e.g.
`{"x": "726", "y": "122", "repeat": "1", "repeat_delay_in_ms": "50"}` for
the Android tap tool), tools failed with opaque errors such as
Tool raised an unexpected error: '<' not supported between instances
of 'str' and 'int'
which made the agent fall back to drag-and-drop workarounds and break test
runs.
Add `coerce_tool_input()`, which walks the tool's JSON schema (refs resolved,
`anyOf`/`oneOf`/type lists, nested objects and arrays) and performs only
lossless conversions: numeric strings and integral floats to `integer`,
numeric strings to `number`, `"true"`/`"false"`/`0`/`1` to `boolean`, and
numbers to `string`. Anything that cannot be converted unambiguously is left
untouched so the tool's own validation still applies. Apply it in
`ToolCollection._run_regular_tool` so every regular tool benefits.
Add unit tests for the coercion itself and a regression test that runs the
real `AndroidTapTool` through `ToolCollection` with the string payload seen
in the failing reports.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
philipph-askui
force-pushed
the
fix/coerce-tool-input-types
branch
from
September 22, 2026 10:12
cb2e151 to
22e87ec
Compare
Address review feedback on the tool input coercion: - Do not convert booleans to strings for `string` fields. Choosing between `"True"` and `"true"` would be a guess, so the value is left for the tool. - Drop `"yes"`/`"no"` from boolean coercion so the accepted spellings match the documented rules (`"true"`/`"false"`, `1`/`0`). - Add a regression test that a non-coercible value still surfaces as an `is_error` tool result through `ToolCollection.run()` instead of escaping the dispatcher. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Three Android test runs from 2026-09-22 failed with
every time the agent called
tap_tool. In all eight occurrences the model (Claude via Bedrock) had emitted the integer arguments as JSON strings:{ "x": "726", "y": "122", "repeat": "1", "repeat_delay_in_ms": "50" }AndroidTapTool.__call__comparesrepeat_delay_in_ms < 0before doing anything else, and"50" < 0raises theTypeError.ToolCollection._run_regular_toolpassed the model's input straight through astool(**tool_input), so the"type": "integer"in the tool'sinput_schemawas only a hint to the model and never enforced. In one run the agent gave up on tapping after four failures, fell back todrag_and_drop_tool, and marked the test BROKEN.Change
Generic coercion in the tool dispatcher. New
askui.models.shared.tool_input_coercion.coerce_tool_input()walks the tool's JSON schema (refs resolved,anyOf/oneOf/type lists, nested objects and arrays) and performs only lossless conversions: numeric strings and integral floats tointeger, numeric strings tonumber,"true"/"false"/0/1toboolean, numbers tostring. Anything ambiguous is left untouched so the tool's own validation still applies. It is wired intoToolCollection._run_regular_tool, which is the single place model-driven tool calls (including cached trajectory replay) go through, so every regular tool is protected, not just the tap tool.The tap tool itself is unchanged. A local coercion there would be redundant: MCP exposure via
to_mcp_toolis already validated by FastMCP/Pydantic in lax mode, and direct Python calls are covered by type hints.Scope note: only regular tools dispatched through
_run_regular_toolare coerced. Tools served by external MCP servers (e.g. Playwright MCP, which validates strictly) still receive the model input unchanged. Extending coercion to_call_mcp_toolis a possible follow-up.Tests
tests/unit/models/shared/test_tool_input_coercion.py: coercion rules, unions, nested objects/arrays,$refresolution, no-op cases, and an end-to-endToolCollection.run()case.tests/unit/tools/android/test_tap_tool.py: basicAndroidTapToolbehaviour plus a regression test running the real tap tool throughToolCollectionwith the exact string payload from the failing reports.Verification
pdm run qa:fixclean (mypy, ruff format, ruff lint)pdm run test:unit: 811 passed🤖 Generated with Claude Code