fix(ai-red-teaming): JSON-encode prompt in generated agent target bodies - #152
Merged
Merged
Conversation
The generated HTTP-agent target built its request body with
`{template}.replace('{prompt}', prompt.replace('"', '\\"'))`, escaping only
double quotes. Multi-line / backslash-heavy adversarial prompts (GOAT, TAP,
crescendo) then produced invalid JSON, so json.loads failed and the target
returned empty responses - attacks recorded 0 tool_calls / empty content and
trials failed even against a vulnerable agent. Substitute the prompt with
json.dumps(prompt)[1:-1] so quotes, newlines, and backslashes are escaped
correctly. Bump 1.17.1 -> 1.17.2.
Found by running a GOAT attack via the TUI against a dict-returning agent that
returns populated tool_calls on a direct curl but empty responses through the
attack.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The generated HTTP-agent target built its request body with
{template}.replace('{prompt}', prompt.replace('"', '\\"'))- escaping onlydouble quotes. Multi-line / backslash-heavy adversarial prompts (GOAT, TAP,
crescendo) then produced invalid JSON, so
json.loads(body_str)failed andthe target returned empty responses. Attacks recorded 0 tool_calls / empty
content and trials failed even against a vulnerable agent.
Found by running a GOAT attack via the TUI against a dict-returning agent that
returns populated
tool_callson a direct curl but empty responses through theattack.
Fix
Substitute the prompt with
json.dumps(prompt)[1:-1]so quotes, newlines, andbackslashes are escaped correctly (valid JSON for any prompt). Applied to both
generated-target builders (agentic + custom). Bump
1.17.1 -> 1.17.2.Validation
TestAgentTargetPromptEscaping(old escaping fails json.loads on a nastymulti-line prompt; new escaping round-trips exactly) + existing normalization
tests -> pass.
'{"message":"{prompt}"}'.replace('{prompt}', json.dumps(nasty)[1:-1])parses and preserves the prompt for a prompt containing newlines, quotes, and
backslashes.