Update ASI02_Tool_Misuse_and_Exploitation .md #730

goriggs · 2025-10-06T13:23:36Z

Update ASI02_Tool_Misuse_and_Exploitation

Summary
First initial draft of ASI02 Tool Misuse and Exploitation

Changes

Added description
Added examples
Added how to prevent
Added example attack scenarios
Added reference links

Impact

This is an update of the ASI02 template for the team to review.

Signed-off-by: goriggs <[email protected]>

itskerenkatz

Beautiful work. Please look at the comments to address conflicts with ASI03 and ASI04. Thanks!

itskerenkatz · 2025-10-07T15:16:14Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI02_Tool_Misuse_and_Exploitation .md

+Authors:  Gaurav M, Riggs Goodman III

-A brief description of the vulnerability that includes its potential effects such as system compromises, data breaches, or other security concerns.
+Agentic AI systems integrate external tools (APIs, plugins, shells, browsers, databases, file systems) as part of their reasoning loops. Tool misuse covers cases where the agent is granted a legitimate, trusted tool but uses it unsafely. If the tool itself is poisoned, impersonated or malicious, see ASI04:Agentic Supply Chain Vulnerabilities.


As far as I know (and correct me if I'm mistaken) it is not only as part of the reasoning, but also agents that do not implement reasoning can recommend on tools invokements.

Also - loved the way you differentiate between ASI04 and this one.

Yes, if the tool definition is included as part of the request to the LLM, it can be used. The system that uses it could be an agent or application, just depending on architecture. How would you want me to change this one?

I'll update it as I take it to the aggregated document if so :)

itskerenkatz · 2025-10-07T15:17:59Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI02_Tool_Misuse_and_Exploitation .md

-1. Example 1: Specific instance or type of this vulnerability.
-2. Example 2: Another instance or type of this vulnerability.
-3. Example 3: Yet another instance or type of this vulnerability.
+1.	Over-Privileged Tool Access – Email summarizer that also has permissions to delete or send messages without user confirmation.


Isn't it more a classic permissions problem rather a unique agentic problem?

It is ABSOLUTELY a permissions problem. But, it gets back to how the system is architected, the permissions given, and how the LLM uses it when you have an agent orchestrating all the different tools as part of the reasoning. If there is a better way to say this or show this, please suggest. But, this is one of the biggest issues with tool use/mis-use, which is why it's 1

itskerenkatz · 2025-10-07T15:18:28Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI02_Tool_Misuse_and_Exploitation .md

-2. Example 2: Another instance or type of this vulnerability.
-3. Example 3: Yet another instance or type of this vulnerability.
+1.	Over-Privileged Tool Access – Email summarizer that also has permissions to delete or send messages without user confirmation.
+2.	Unvalidated Input Forwarding – Agent passes untrusted model output directly to a shell executor (rm -rf /).


directly to an external tool (for example, shell executor), right?

itskerenkatz · 2025-10-07T15:19:03Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI02_Tool_Misuse_and_Exploitation .md

+2.	Unvalidated Input Forwarding – Agent passes untrusted model output directly to a shell executor (rm -rf /).
+3.	Unsafe Browsing / Federated Calls – Research agent clicks malicious links, downloads malware, or executes hidden prompt instructions.
+4.	Loop Amplification – Planning agents repeatedly invoke billing-heavy APIs, causing denial of service or cost exhaustion.
+5.	Cross-Agent Confused Deputy – One agent convinces another to execute privileged tasks (multi-agent exploitation).


Again feels to me more of an identity problem (ASI03)

I think the focus of this one is not escalation of privileges, but selecting the wrong permissions in the first place

the way we thought about this from below scenario

was agent A able to act as a different principal (fake token, spoofed identity, forged signature, or swapped provenance)?
this would come under -> Identity / Trust boundary issue (e.g., “Cross-agent impersonation / identity confusion”).

Or did agent A simply persuade agent B to run an operation B was already authorized to run (social engineering, prompt injection, no identity spoofing)?
this would come under -> Tool misuse (e.g., “Cross-agent confused deputy — privileged tool misuse”).

If both happened like A both spoofed identity and induced B to use its tool :)
then you can also combine them as -> identity breach leading to tool misuse (make identity the primary cause if spoofing enabled the misuse).

itskerenkatz · 2025-10-07T15:19:16Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI02_Tool_Misuse_and_Exploitation .md

+3.	Unsafe Browsing / Federated Calls – Research agent clicks malicious links, downloads malware, or executes hidden prompt instructions.
+4.	Loop Amplification – Planning agents repeatedly invoke billing-heavy APIs, causing denial of service or cost exhaustion.
+5.	Cross-Agent Confused Deputy – One agent convinces another to execute privileged tasks (multi-agent exploitation).
+6.	Tool Chaining (Policy Bypass)– Indirect access to restricted functions or exfiltration via sequential tool calls. Eg: The agent composes allowed tools so the combined effect violates policy (DB‑read -> CSV‑export -> email‑sent to external domain).


Agree, we can remove this one

itskerenkatz · 2025-10-07T15:21:12Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI02_Tool_Misuse_and_Exploitation .md

+6.	Tool Chaining (Policy Bypass)– Indirect access to restricted functions or exfiltration via sequential tool calls. Eg: The agent composes allowed tools so the combined effect violates policy (DB‑read -> CSV‑export -> email‑sent to external domain).
+7.	External Data Tool Poisoning – Agent consumes malicious/misleading content from third‑party sources (web, email, files, memory, services), which then steers unsafe tool actions.
+8.	Tool Squatting:A specialized instantiation of this risk, Tool Squatting, describes a malevolent tactic whereby an adversary misleads agents or exploits automated discovery mechanisms through the surreptitious registration, nomenclature, or presentation of a malicious tool, capability, or API. This deceptive maneuver induces agents to establish interaction with the adversarial entity, thereby facilitating the compromise of their operational integrity or the broader system's security posture. 



What about the classic example you have mentioned in the intro -
An agent getting the wrong (not even adversarial) context (from browsing online or from RAG/ knowledge) and then execute the right tool function call with the wrong parameters and effects the operations of the company? (classic example: editing accounting excel file with the wrong income value)

Is this in addition to 8 or replacement of 8?

itskerenkatz · 2025-10-07T15:21:19Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI02_Tool_Misuse_and_Exploitation .md

-1. Prevention Step 1: A step or strategy that can be used to prevent the vulnerability or mitigate its effects.
-2. Prevention Step 2: Another prevention step or strategy.
-3. Prevention Step 3: Yet another prevention step or strategy.
+1.	Least Privilege for Tools – Limit scope and permissions (e.g., read-only DB queries, no send/delete for email summarizers).


Somehow I think this should be in here, maybe with reference to ASI03.

itskerenkatz · 2025-10-07T15:23:04Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI02_Tool_Misuse_and_Exploitation .md

+[Building AI Agents with Python: From LangChain to AutoGPT](https://python.plainenglish.io/building-ai-agents-with-python-from-langchain-to-autogpt-925f82463645)
+
+Scenario 6: Multi-Agent Abuse
+In an A2A network, a malicious helper agent convinces the orchestrator to forward sensitive tickets by overstating tool capabilities.


Isn't it more ASI04 as you beautifully stated in the intro?

True, we can remove this one

Updated based on feedback Signed-off-by: goriggs <[email protected]>

xycloops123 · 2025-10-14T15:16:13Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI02_Tool_Misuse_and_Exploitation .md

 ## ASI02 – Tool Misuse and Exploitation

 **Description:**
+Authors:  Gaurav M, Riggs Goodman III


Suggested change

Authors: Gaurav M, Riggs Goodman III

Authors: Gaurav Mukherjee, Riggs Goodman III

Update ASI02_Tool_Misuse_and_Exploitation .md

600a373

Signed-off-by: goriggs <[email protected]>

goriggs requested review from guerilla7, hoeg and itskerenkatz as code owners October 6, 2025 13:23

itskerenkatz requested changes Oct 7, 2025

View reviewed changes

Update ASI02_Tool_Misuse_and_Exploitation .md

64a6e5e

Updated based on feedback Signed-off-by: goriggs <[email protected]>

xycloops123 reviewed Oct 14, 2025

View reviewed changes

	Authors: Gaurav M, Riggs Goodman III
	Authors: Gaurav Mukherjee, Riggs Goodman III

Uh oh!

Update ASI02_Tool_Misuse_and_Exploitation .md #730

Are you sure you want to change the base?

Update ASI02_Tool_Misuse_and_Exploitation .md #730

Uh oh!

Conversation

goriggs commented Oct 6, 2025

Update ASI02_Tool_Misuse_and_Exploitation

Uh oh!

itskerenkatz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants