Skip to content

fix(tools): coerce string-typed integer tool arguments (Android tap tool error) - #317

Merged
philipph-askui merged 2 commits into
mainfrom
fix/coerce-tool-input-types
Sep 22, 2026
Merged

philipph-askui merged 2 commits into
mainfrom
fix/coerce-tool-input-types

Conversation

@philipph-askui

@philipph-askui philipph-askui commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Three Android test runs from 2026-09-22 failed with

Tool raised an unexpected error: '<' not supported between instances of 'str' and 'int'

every time the agent called tap_tool. In all eight occurrences the model (Claude via Bedrock) had emitted the integer arguments as JSON strings:

{ "x": "726", "y": "122", "repeat": "1", "repeat_delay_in_ms": "50" }

AndroidTapTool.__call__ compares repeat_delay_in_ms < 0 before doing anything else, and "50" < 0 raises the TypeError. ToolCollection._run_regular_tool passed the model's input straight through as tool(**tool_input), so the "type": "integer" in the tool's input_schema was only a hint to the model and never enforced. In one run the agent gave up on tapping after four failures, fell back to drag_and_drop_tool, and marked the test BROKEN.

Change

Generic coercion in the tool dispatcher. New askui.models.shared.tool_input_coercion.coerce_tool_input() walks the tool's JSON schema (refs resolved, anyOf/oneOf/type lists, nested objects and arrays) and performs only lossless conversions: numeric strings and integral floats to integer, numeric strings to number, "true"/"false"/0/1 to boolean, numbers to string. Anything ambiguous is left untouched so the tool's own validation still applies. It is wired into ToolCollection._run_regular_tool, which is the single place model-driven tool calls (including cached trajectory replay) go through, so every regular tool is protected, not just the tap tool.

The tap tool itself is unchanged. A local coercion there would be redundant: MCP exposure via to_mcp_tool is already validated by FastMCP/Pydantic in lax mode, and direct Python calls are covered by type hints.

Scope note: only regular tools dispatched through _run_regular_tool are coerced. Tools served by external MCP servers (e.g. Playwright MCP, which validates strictly) still receive the model input unchanged. Extending coercion to _call_mcp_tool is a possible follow-up.

Tests

  • tests/unit/models/shared/test_tool_input_coercion.py: coercion rules, unions, nested objects/arrays, $ref resolution, no-op cases, and an end-to-end ToolCollection.run() case.
  • tests/unit/tools/android/test_tap_tool.py: basic AndroidTapTool behaviour plus a regression test running the real tap tool through ToolCollection with the exact string payload from the failing reports.

Verification

  • pdm run qa:fix clean (mypy, ruff format, ruff lint)
  • pdm run test:unit: 811 passed

🤖 Generated with Claude Code

…types

`ToolCollection` passed the model's tool arguments straight into the tool
implementation without looking at the tool's `input_schema`. When a model
emitted integers as JSON strings (observed with Claude via Bedrock, e.g.
`{"x": "726", "y": "122", "repeat": "1", "repeat_delay_in_ms": "50"}` for
the Android tap tool), tools failed with opaque errors such as

    Tool raised an unexpected error: '<' not supported between instances
    of 'str' and 'int'

which made the agent fall back to drag-and-drop workarounds and break test
runs.

Add `coerce_tool_input()`, which walks the tool's JSON schema (refs resolved,
`anyOf`/`oneOf`/type lists, nested objects and arrays) and performs only
lossless conversions: numeric strings and integral floats to `integer`,
numeric strings to `number`, `"true"`/`"false"`/`0`/`1` to `boolean`, and
numbers to `string`. Anything that cannot be converted unambiguously is left
untouched so the tool's own validation still applies. Apply it in
`ToolCollection._run_regular_tool` so every regular tool benefits.

Add unit tests for the coercion itself and a regression test that runs the
real `AndroidTapTool` through `ToolCollection` with the string payload seen
in the failing reports.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@philipph-askui
philipph-askui force-pushed the fix/coerce-tool-input-types branch from cb2e151 to 22e87ec Compare September 22, 2026 10:12
Address review feedback on the tool input coercion:

- Do not convert booleans to strings for `string` fields. Choosing between
  `"True"` and `"true"` would be a guess, so the value is left for the tool.
- Drop `"yes"`/`"no"` from boolean coercion so the accepted spellings match
  the documented rules (`"true"`/`"false"`, `1`/`0`).
- Add a regression test that a non-coercible value still surfaces as an
  `is_error` tool result through `ToolCollection.run()` instead of escaping
  the dispatcher.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@philipph-askui
philipph-askui merged commit db18b0c into main Sep 22, 2026
1 check passed
@philipph-askui
philipph-askui deleted the fix/coerce-tool-input-types branch September 22, 2026 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant