Update ASI03_Identity_and_Privilege_Abuse .md #722

KellenCarl · 2025-09-22T04:13:12Z

Updated Ken Huang's initial draft with my comments and created updates based on Keren Katz' feedback on Update ASI03_Privilege_Compromise.md #716

[Title of Your PR]

Key Changes:

List major changes and core updates
Keep each line under 80 characters
Focus on the "what" and "why"

Added:

New features/functionality
New files/configurations
New dependencies

Changed:

Updates to existing code
Configuration changes
Dependency updates

Removed:

Deleted files/code
Removed dependencies
Cleaned up configurations

Updated Ken Huang's initial draft with my comments and created updates based on Keren Katz' feedback on Update ASI03_Privilege_Compromise.md OWASP#716 Signed-off-by: Kellen Carl <[email protected]>

kerenkatzapex

AMAZING AMAZING WORK.

kerenkatzapex · 2025-09-29T06:26:51Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI03_Identity_and_Privilege_Abuse .md

 **Description:**

-A brief description of the vulnerability that includes its potential effects such as system compromises, data breaches, or other security concerns.
+Identity & Privilege Abuse is the exploitation of dynamic trust relationships and delegation chains between autonomous AI agents to escalate privileges and bypass security controls. This risk introduces fundamentally new attack vectors that traditional security models don't address. Unlike static privilege escalation, attackers exploit the dynamic delegation chains that agents create autonomously—manipulating how AI agents inherit, transfer, and act on permissions across interconnected systems. The vulnerability can occur through the direct manipulation of permissions, exploitation of role inheritance, hijacking of control systems, or exploitation of the agent's context (e.g., memory, conversation history). An attacker can compromise one agent and use it as a pivot point to trick a more privileged agent into executing unauthorized actions, effectively bypassing security controls.


Add context to memory and conversation history

kerenkatzapex · 2025-09-29T06:29:33Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI03_Identity_and_Privilege_Abuse .md

+
+4. **Delegation Chain Abuse & Control-Flow Hijacking:** This is a sophisticated attack that targets the orchestration logic of a multi-step agentic workflow. An attacker injects a malicious instruction or manipulates metadata (like task plans or action histories) early in a delegation chain. As the task is passed from agent to agent, the malicious payload is carried along, trusted by each subsequent agent. The goal is to have a downstream agent, which may have much higher privileges, execute a high-impact, unauthorized action. Control-flow hijacking is a variant where the manipulated metadata doesn't just alter an action but redirects the entire task to an attacker-controlled agent or tool.
+
+5. **Credential and Token Mismanagement:** This risk covers the insecure handling of the secrets that grant agents their power. It can manifest in several ways: developers hardcoding API keys in agent code, agents storing secrets insecurely in logs or memory, or agents being manipulated via prompt injection to reveal their own credentials. Unlike other risks that abuse an agent's actions, this one targets the agent's identity itself. The compromise of a single, long-lived token can give an attacker direct, persistent, and often untraceable access to underlying APIs and systems.


A suggestion: I love to call it Access data abuse, to cover all types of access data :)

kerenkatzapex · 2025-09-29T06:32:27Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI03_Identity_and_Privilege_Abuse .md

+   - Risks Mitigated: Credential and Token Mismanagement (Risk #5), Identity Forgery and Shadow Bridging (Risk #7)
+   - Scenario Mitigation: This directly prevents the vulnerability in the Memory-Based Escalation (Scenario #2), as credentials would be fetched dynamically from a vault for the specific SSH action and never stored in the agent's memory. It also helps mitigate the BYO-AI Connector (Scenario #6) by treating the third-party app's identity as distinct and applying a much stricter access policy to it.
+
+6. **Validate Control-Flow and Third-Party Dependencies:** Secure the metadata that guides agent orchestration (e.g., task plans) to prevent hijacking. This requires implementing a mature vendor risk management program to enforce a strict approval process for integrating third-party AI connectors. The process must include a careful scrutinization of all requested OAuth scopes to prevent "bring-your-own-AI" tools from gaining excessive permissions. While this can be difficult to enforce in organizations that prioritize developer speed, it is critical for preventing supply chain and integration-based attacks.


Can the hijacking source be only in third party tools? I think here we need to refer to context injection and to talk about validating any input to the LLM (either context or third party)

Added in an explicit call-out to first party, or other "trusted" context.

Edits to improve clarity and simplicity of communication. Removed Scenarios so readers can focus specifically on risks and mitigations. Many of the risk definitions are clearly stated and will lead users to imagine the scenarios listed, or seek further information via the references/appendix. Signed-off-by: Kellen Carl <[email protected]>

KellenCarl · 2025-10-03T03:06:04Z

@kerenkatzapex, further edits made to improve clarity and cohesive messaging.
Removed some information that is covered by other entries.

kerenkatzapex

We should get the Example Attack Scenarios part back and make each of the parts more concise :)

kerenkatzapex · 2025-10-05T08:01:32Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI03_Identity_and_Privilege_Abuse .md

+Identity & Privilege Abuse is the exploitation of dynamic trust relationships and delegation chains between autonomous AI agents to escalate privileges and bypass security controls. Unlike static privilege escalation, attackers exploit the dynamic delegation chains that agents create autonomously manipulating how AI agents inherit, transfer, and act on permissions across interconnected systems. The vulnerability can occur through the direct manipulation of permissions, exploitation of role inheritance, hijacking of control systems, or exploitation of the agent's context (e.g., memory, conversation history). 

-**Common Examples of Vulnerability:**
+**Common Examples of Risk:**


It's important to stick to the titles we have in the template to have consistency :)

Reverted to original to keep consistent with overall Top 10 Document.

kerenkatzapex · 2025-10-05T08:03:27Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI03_Identity_and_Privilege_Abuse .md

+
+6. **Apply Human-in-the-Loop for Privilege Escalation:** For any action that requires an agent to gain higher privileges or perform a high-impact, irreversible action (e.g., creating an admin account, deploying code, processing a large payment), mandate explicit approval from a human user. Implementation of this control involves identifying these critical operations and forcing them into a queue that requires human sign-off. While this can create friction and slow down automated processes, it serves as an essential final safeguard against catastrophic failure.
+   - Risks Mitigated: A crucial backstop for the most severe outcomes of nearly all listed risks.



Missing: the entire Example Attack Scenarios part
We should bring it back please

I have added these back.

kerenkatzapex · 2025-10-05T08:04:27Z

...e/agentic-top-10/Sprint 1-first-public-draft-expanded/ASI03_Identity_and_Privilege_Abuse .md

+
+**Appendix:**
+
+**Mapping to OWASP Top 10 for LLMs**


I think ideally:

Having 1 sentence for each as part of the intro at the beginning

Adding the rest to the general appendix file that we have

Added another paragraph to intro, tying ASI 03 to LLM top 10.
I have not found the best way to do this with AIVSS and Threats and Mitigations.
In some ways I think the appendix may be most appropriate.

Add back Scenarios and Scenario Mitigations. Signed-off-by: Kellen Carl <[email protected]>

Further editing for clarity. Signed-off-by: Kellen Carl <[email protected]>

Added introductory section highlighting the risks and outcomes of successful exploitation of Identity and Privilege Abuse, and tied in to OWASP LLM top 10 Document. Signed-off-by: Kellen Carl <[email protected]>

Update ASI03_Identity_and_Privilege_Abuse .md

e0ddc02

Updated Ken Huang's initial draft with my comments and created updates based on Keren Katz' feedback on Update ASI03_Privilege_Compromise.md OWASP#716 Signed-off-by: Kellen Carl <[email protected]>

KellenCarl requested review from guerilla7, hoeg and itskerenkatz as code owners September 22, 2025 04:13

kerenkatzapex reviewed Sep 29, 2025

View reviewed changes

kerenkatzapex suggested changes Oct 5, 2025

View reviewed changes

KellenCarl added 3 commits October 5, 2025 10:15

Update ASI03_Identity_and_Privilege_Abuse .md

78beb97

Add back Scenarios and Scenario Mitigations. Signed-off-by: Kellen Carl <[email protected]>

Update ASI03_Identity_and_Privilege_Abuse .md

0d186f3

Further editing for clarity. Signed-off-by: Kellen Carl <[email protected]>

Update ASI03_Identity_and_Privilege_Abuse .md

c3ab901

Added introductory section highlighting the risks and outcomes of successful exploitation of Identity and Privilege Abuse, and tied in to OWASP LLM top 10 Document. Signed-off-by: Kellen Carl <[email protected]>


		4. Delegation Chain Abuse & Control-Flow Hijacking: This is a sophisticated attack that targets the orchestration logic of a multi-step agentic workflow. An attacker injects a malicious instruction or manipulates metadata (like task plans or action histories) early in a delegation chain. As the task is passed from agent to agent, the malicious payload is carried along, trusted by each subsequent agent. The goal is to have a downstream agent, which may have much higher privileges, execute a high-impact, unauthorized action. Control-flow hijacking is a variant where the manipulated metadata doesn't just alter an action but redirects the entire task to an attacker-controlled agent or tool.

		5. Credential and Token Mismanagement: This risk covers the insecure handling of the secrets that grant agents their power. It can manifest in several ways: developers hardcoding API keys in agent code, agents storing secrets insecurely in logs or memory, or agents being manipulated via prompt injection to reveal their own credentials. Unlike other risks that abuse an agent's actions, this one targets the agent's identity itself. The compromise of a single, long-lived token can give an attacker direct, persistent, and often untraceable access to underlying APIs and systems.


		6. Apply Human-in-the-Loop for Privilege Escalation: For any action that requires an agent to gain higher privileges or perform a high-impact, irreversible action (e.g., creating an admin account, deploying code, processing a large payment), mandate explicit approval from a human user. Implementation of this control involves identifying these critical operations and forcing them into a queue that requires human sign-off. While this can create friction and slow down automated processes, it serves as an essential final safeguard against catastrophic failure.
		- Risks Mitigated: A crucial backstop for the most severe outcomes of nearly all listed risks.

Uh oh!

Update ASI03_Identity_and_Privilege_Abuse .md #722

Are you sure you want to change the base?

Update ASI03_Identity_and_Privilege_Abuse .md #722

Uh oh!

Conversation

KellenCarl commented Sep 22, 2025

[Title of Your PR]

Uh oh!

kerenkatzapex left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KellenCarl commented Oct 3, 2025

Uh oh!

kerenkatzapex left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants