T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search
Abstract
T-MAP, a trajectory-aware evolutionary search method, discovers adversarial prompts that bypass safety measures and achieve harmful outcomes through tool interactions in LLM agents.
While prior red-teaming efforts have focused on eliciting harmful text outputs from large language models (LLMs), such approaches fail to capture agent-specific vulnerabilities that emerge through multi-step tool execution, particularly in rapidly growing ecosystems such as the Model Context Protocol (MCP). To address this gap, we propose a trajectory-aware evolutionary search method, T-MAP, which leverages execution trajectories to guide the discovery of adversarial prompts. Our approach enables the automatic generation of attacks that not only bypass safety guardrails but also reliably realize harmful objectives through actual tool interactions. Empirical evaluations across diverse MCP environments demonstrate that T-MAP substantially outperforms baselines in attack realization rate (ARR) and remains effective against frontier models, including GPT-5.2, Gemini-3-Pro, Qwen3.5, and GLM-5, thereby revealing previously underexplored vulnerabilities in autonomous LLM agents.
Community
Red-teaming tool-calling agents
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks (2026)
- LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios (2026)
- TrailBlazer: History-Guided Reinforcement Learning for Black-Box LLM Jailbreaking (2026)
- RvB: Automating AI System Hardening via Iterative Red-Blue Games (2026)
- Visual Exclusivity Attacks: Automatic Multimodal Red Teaming via Agentic Planning (2026)
- Learning to Inject: Automated Prompt Injection via Reinforcement Learning (2026)
- AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2603.22341 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper