Skip to content

Tools: Keep edge whitespace in parsed tool arguments - #524

Open
anubhavlal07 wants to merge 1 commit into
theroyallab:mainfrom
anubhavlal07:fix/preserve-tool-arg-whitespace
Open

anubhavlal07 wants to merge 1 commit into
theroyallab:mainfrom
anubhavlal07:fix/preserve-tool-arg-whitespace

Conversation

@anubhavlal07

Copy link
Copy Markdown

Is your pull request related to a problem? Please describe.
Fixes #523.

Parsed tool-call arguments lose their leading and trailing whitespace. coerce_param_value() in toolcall_formats/common.py called raw.strip(), although its docstring says only the newlines the template adds around a value are removed. The qwen3_coder, minimax_m2, glm4_5 and hy3 parsers also called .strip() on each value before coercing it. The delta streamer in toolcall_stream.py did the same so that its output matched those parsers. As a result a string argument " SPACED2 " came back as "SPACED2", and code passed to edit/write tools lost the indentation on its first line.

Why should this feature be added?
Leading and trailing whitespace in a string argument is part of the value: an indented code block, a padded search string for an edit tool, and so on. Only the template's own \n wrappers should be removed.

The change:

  • A new strip_template_newlines() in common.py (raw.strip("\r\n")). coerce_param_value() uses it. A .strip() copy is still used for the empty check and the json.loads attempt, so " 7 " with type integer still becomes 7. The plain-string fallback returns the newline-stripped value.
  • The four parsers pass the raw value to coerce_param_value() and no longer strip it first.
  • ToolCallDeltaEmitter.value_text and close_param use the same helper, so streamed and end-of-stream arguments stay byte-identical. _is_streamable still gets the fully stripped text. Trailing newlines are held back until later text makes them interior, the same way trailing whitespace was held back before. I updated the module docstring and the inline comment to match.

The GLM-4.5 and Hy3 templates render <arg_value> without newlines around the value, so for them the change just stops removing whitespace the model wrote.

Examples
<parameter=path>\n SPACED2 \n</parameter> with path: {"type": "string"}:

  • before: {"path": "SPACED2"}
  • after: {"path": " SPACED2 "}

Additional context
New tests in tests/test_tool_param_types.py:

  • test_coercion_keeps_edge_whitespace: string, untyped, integer, object and all-whitespace values.
  • test_edge_whitespace_survives_parsing: qwen3_coder with a padded string, a tab-indented code block and padded integer/object values, plus minimax_m2, glm4_5 and hy3.
  • test_edge_whitespace_streams_like_the_parse: the joined QwenToolCallDeltaStreamer deltas equal the parser's arguments at chunk sizes 1, 3 and 7, with and without a schema, and no "diverged" error is logged.

The three new tests fail without the fix and pass with it. test_tool_param_types.py, test_toolcall_stream.py, test_tool_delta_streaming.py and test_stream_parser.py all pass, and ruff check and format are clean. #522 also edits coerce_param_value(), so whichever lands second will need a small rebase.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] edge whitespace lost in parsed tool arguments

1 participant