fix(integrations): ensure that GEN_AI_AGENT_NAME is properly set for GEN_AI spans under an invoke_agent span #5030

constantinius · 2025-10-24T13:44:45Z

Description

chat and execute tool spans under an invoke agent span should have the agent name property set

Issues

Closes https://linear.app/getsentry/issue/TET-1293/make-sure-that-agent-name-is-set-on-all-of-its-gen-ai-children

…GEN_AI spans under an invoke_agent span

linear · 2025-10-24T13:44:49Z

TET-1293 Make sure that agent name is set on all of its gen_ai.* children

codecov · 2025-10-24T19:51:27Z

❌ 12 Tests Failed:

Tests completed	Failed	Passed	Skipped
27783	12	27771	2223

View the top 3 failed test(s) by shortest run time

tests.integrations.langchain.test_langchain::test_langchain_agent[True-False-False]

Stack Traces | 0.301s run time

.../integrations/langchain/test_langchain.py:204: in test_langchain_agent
    assert "gen_ai.usage.input_tokens" in chat_spans[0]["data"]
E   assert 'gen_ai.usage.input_tokens' in {'thread.id': '139760444132224', 'thread.name': 'MainThread', 'gen_ai.operation.name': 'chat', 'gen_ai.request.model': 'gpt-3.5-turbo', 'gen_ai.system': 'openai', 'gen_ai.request.temperature': 0.0, 'gen_ai.request.available_tools': '[{"name": "get_word_length", "description": "get_word_length(word: str) -> int - Returns the length of a word."}]', 'gen_ai.response.finish_reasons': 'function_call'}

tests.integrations.langchain.test_langchain::test_langchain_agent[False-False-True]

Stack Traces | 0.303s run time

.../integrations/langchain/test_langchain.py:204: in test_langchain_agent
    assert "gen_ai.usage.input_tokens" in chat_spans[0]["data"]
E   assert 'gen_ai.usage.input_tokens' in {'thread.id': '139945438190464', 'thread.name': 'MainThread', 'gen_ai.operation.name': 'chat', 'gen_ai.request.model': 'gpt-3.5-turbo', 'gen_ai.request.temperature': 0.0, 'gen_ai.request.available_tools': '[{"name": "get_word_length", "description": "get_word_length(word: str) -> int - Returns the length of a word."}]', 'gen_ai.response.finish_reasons': 'function_call'}

tests.integrations.langchain.test_langchain::test_langchain_agent[True-False-False]

Stack Traces | 0.303s run time

.../integrations/langchain/test_langchain.py:204: in test_langchain_agent
    assert "gen_ai.usage.input_tokens" in chat_spans[0]["data"]
E   assert 'gen_ai.usage.input_tokens' in {'thread.id': '139945438190464', 'thread.name': 'MainThread', 'gen_ai.operation.name': 'chat', 'gen_ai.request.model': 'gpt-3.5-turbo', 'gen_ai.system': 'openai', 'gen_ai.request.temperature': 0.0, 'gen_ai.request.available_tools': '[{"name": "get_word_length", "description": "get_word_length(word: str) -> int - Returns the length of a word."}]', 'gen_ai.response.finish_reasons': 'function_call'}

tests.integrations.langchain.test_langchain::test_langchain_agent[False-True-False]

Stack Traces | 0.304s run time

.../integrations/langchain/test_langchain.py:204: in test_langchain_agent
    assert "gen_ai.usage.input_tokens" in chat_spans[0]["data"]
E   assert 'gen_ai.usage.input_tokens' in {'thread.id': '139945438190464', 'thread.name': 'MainThread', 'gen_ai.operation.name': 'chat', 'gen_ai.request.model': 'gpt-3.5-turbo', 'gen_ai.system': 'openai', 'gen_ai.request.temperature': 0.0, 'gen_ai.request.available_tools': '[{"name": "get_word_length", "description": "get_word_length(word: str) -> int - Returns the length of a word."}]', 'gen_ai.response.finish_reasons': 'function_call'}

tests.integrations.langchain.test_langchain::test_langchain_agent[False-False-True]

Stack Traces | 0.306s run time

.../integrations/langchain/test_langchain.py:204: in test_langchain_agent
    assert "gen_ai.usage.input_tokens" in chat_spans[0]["data"]
E   assert 'gen_ai.usage.input_tokens' in {'thread.id': '139760444132224', 'thread.name': 'MainThread', 'gen_ai.operation.name': 'chat', 'gen_ai.request.model': 'gpt-3.5-turbo', 'gen_ai.request.temperature': 0.0, 'gen_ai.request.available_tools': '[{"name": "get_word_length", "description": "get_word_length(word: str) -> int - Returns the length of a word."}]', 'gen_ai.response.finish_reasons': 'function_call'}

tests.integrations.langchain.test_langchain::test_langchain_agent[False-False-True]

Stack Traces | 0.338s run time

.../integrations/langchain/test_langchain.py:204: in test_langchain_agent
    assert "gen_ai.usage.input_tokens" in chat_spans[0]["data"]
E   assert 'gen_ai.usage.input_tokens' in {'thread.id': '140286610340736', 'thread.name': 'MainThread', 'gen_ai.operation.name': 'chat', 'gen_ai.request.model': 'gpt-3.5-turbo', 'gen_ai.request.temperature': 0.0, 'gen_ai.request.available_tools': '[{"name": "get_word_length", "description": "get_word_length(word: str) -> int - Returns the length of a word."}]', 'gen_ai.response.finish_reasons': 'function_call'}

tests.integrations.langchain.test_langchain::test_langchain_agent[True-False-False]

Stack Traces | 0.338s run time

.../integrations/langchain/test_langchain.py:204: in test_langchain_agent
    assert "gen_ai.usage.input_tokens" in chat_spans[0]["data"]
E   assert 'gen_ai.usage.input_tokens' in {'thread.id': '140286610340736', 'thread.name': 'MainThread', 'gen_ai.operation.name': 'chat', 'gen_ai.request.model': 'gpt-3.5-turbo', 'gen_ai.system': 'openai', 'gen_ai.request.temperature': 0.0, 'gen_ai.request.available_tools': '[{"name": "get_word_length", "description": "get_word_length(word: str) -> int - Returns the length of a word."}]', 'gen_ai.response.finish_reasons': 'function_call'}

tests.integrations.langchain.test_langchain::test_langchain_agent[False-True-False]

Stack Traces | 0.341s run time

.../integrations/langchain/test_langchain.py:204: in test_langchain_agent
    assert "gen_ai.usage.input_tokens" in chat_spans[0]["data"]
E   assert 'gen_ai.usage.input_tokens' in {'thread.id': '140286610340736', 'thread.name': 'MainThread', 'gen_ai.operation.name': 'chat', 'gen_ai.request.model': 'gpt-3.5-turbo', 'gen_ai.system': 'openai', 'gen_ai.request.temperature': 0.0, 'gen_ai.request.available_tools': '[{"name": "get_word_length", "description": "get_word_length(word: str) -> int - Returns the length of a word."}]', 'gen_ai.response.finish_reasons': 'function_call'}

tests.integrations.langchain.test_langchain::test_langchain_agent[False-True-False]

Stack Traces | 0.384s run time

.../integrations/langchain/test_langchain.py:204: in test_langchain_agent
    assert "gen_ai.usage.input_tokens" in chat_spans[0]["data"]
E   assert 'gen_ai.usage.input_tokens' in {'thread.id': '139760444132224', 'thread.name': 'MainThread', 'gen_ai.operation.name': 'chat', 'gen_ai.request.model': 'gpt-3.5-turbo', 'gen_ai.system': 'openai', 'gen_ai.request.temperature': 0.0, 'gen_ai.request.available_tools': '[{"name": "get_word_length", "description": "get_word_length(word: str) -> int - Returns the length of a word."}]', 'gen_ai.response.finish_reasons': 'function_call'}

tests.integrations.langchain.test_langchain::test_langchain_agent[True-True-False]

Stack Traces | 1.14s run time

.../integrations/langchain/test_langchain.py:204: in test_langchain_agent
    assert "gen_ai.usage.input_tokens" in chat_spans[0]["data"]
E   assert 'gen_ai.usage.input_tokens' in {'thread.id': '139945438190464', 'thread.name': 'MainThread', 'gen_ai.operation.name': 'chat', 'gen_ai.request.model': 'gpt-3.5-turbo', 'gen_ai.system': 'openai', 'gen_ai.request.temperature': 0.0, 'gen_ai.request.available_tools': '[{"name": "get_word_length", "description": "get_word_length(word: str) -> int - Returns the length of a word."}]', 'gen_ai.request.messages': '[{"role": "system", "content": "You are very powerful assistant, but don\'t know current events"}, {"role": "user", "content": "How many letters in the word eudca"}]', 'gen_ai.response.finish_reasons': 'function_call', 'gen_ai.response.tool_calls': '[{"name": "get_word_length", "args": {"word": "eudca"}, "id": "call_BbeyNhCKa6kYLYzrD40NGm3b"}]', 'gen_ai.response.text': '5'}

tests.integrations.langchain.test_langchain::test_langchain_agent[True-True-False]

Stack Traces | 1.14s run time

.../integrations/langchain/test_langchain.py:204: in test_langchain_agent
    assert "gen_ai.usage.input_tokens" in chat_spans[0]["data"]
E   assert 'gen_ai.usage.input_tokens' in {'thread.id': '139760444132224', 'thread.name': 'MainThread', 'gen_ai.operation.name': 'chat', 'gen_ai.request.model': 'gpt-3.5-turbo', 'gen_ai.system': 'openai', 'gen_ai.request.temperature': 0.0, 'gen_ai.request.available_tools': '[{"name": "get_word_length", "description": "get_word_length(word: str) -> int - Returns the length of a word."}]', 'gen_ai.request.messages': '[{"role": "system", "content": "You are very powerful assistant, but don\'t know current events"}, {"role": "user", "content": "How many letters in the word eudca"}]', 'gen_ai.response.finish_reasons': 'function_call', 'gen_ai.response.tool_calls': '[{"name": "get_word_length", "args": {"word": "eudca"}, "id": "call_BbeyNhCKa6kYLYzrD40NGm3b"}]', 'gen_ai.response.text': '5'}

tests.integrations.langchain.test_langchain::test_langchain_agent[True-True-False]

Stack Traces | 1.41s run time

.../integrations/langchain/test_langchain.py:204: in test_langchain_agent
    assert "gen_ai.usage.input_tokens" in chat_spans[0]["data"]
E   assert 'gen_ai.usage.input_tokens' in {'thread.id': '140286610340736', 'thread.name': 'MainThread', 'gen_ai.operation.name': 'chat', 'gen_ai.request.model': 'gpt-3.5-turbo', 'gen_ai.system': 'openai', 'gen_ai.request.temperature': 0.0, 'gen_ai.request.available_tools': '[{"name": "get_word_length", "description": "get_word_length(word: str) -> int - Returns the length of a word."}]', 'gen_ai.request.messages': '[{"role": "system", "content": "You are very powerful assistant, but don\'t know current events"}, {"role": "user", "content": "How many letters in the word eudca"}]', 'gen_ai.response.finish_reasons': 'function_call', 'gen_ai.response.tool_calls': '[{"name": "get_word_length", "args": {"word": "eudca"}, "id": "call_BbeyNhCKa6kYLYzrD40NGm3b"}]', 'gen_ai.response.text': '5'}

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

### Description Cherry-picked off of #5030 #### Issues Ref https://linear.app/getsentry/issue/TET-1293/make-sure-that-agent-name-is-set-on-all-of-its-gen-ai-children #### Reminders - Please add tests to validate your changes, and lint your code using `tox -e linters`. - Add GH Issue ID _&_ Linear ID (if applicable) - PR title should use [conventional commit](https://develop.sentry.dev/engineering-practices/commit-messages/#type) style (`feat:`, `fix:`, `ref:`, `meta:`) - For external contributors: [CONTRIBUTING.md](https://github.com/getsentry/sentry-python/blob/master/CONTRIBUTING.md), [Sentry SDK development docs](https://develop.sentry.dev/sdk/), [Discord community](https://discord.gg/Ww9hbqr) --------- Co-authored-by: Fabian Schindler <[email protected]>

…i-agent-name

sentrivana

I took the google-genai part out of this PR and merged it separately since it was straightforward: #5038

Regarding the Langchain part, we talked about this and I also thought about it a bit more. Documenting my reasoning and the proposed path forward for this sort of pattern here.

The problem: We need access to some package-internal object (in this case the agent) to be able to extract something from it in different places in the integration.

In this PR, we're opting for storing it on the current scope in the _contexts. I'm unhappy about this since we're misusing a field whose purpose is something entirely different (populating contexts on events) to store something integration specific.

Other solutions that have been proposed:

Store it on something coming from Langchain directly that gets passed around and is accessible everywhere we need it. This is the usual way of doing things, but we couldn't find a good candidate here.
Store it on the parent transaction and access it from there. The problem here is there might be different agents in a transaction.
Store it on a parent span and access it from here. But there is no good way of traversing the span tree.

TL;DR: recommended way forward:

My recommended solution would be to store this in a ContextVar on the top level of the integration. That way, we have the same execution context isolation that we have with scopes (they're also context vars), but we're not misusing _contexts, and we're keeping the change contained to the integration.

Alternatively, can this be filled in in relay? I.e., we set the agent name on the topmost span that makes sense, and relay copies it to all child spans where it makes sense.

### Description Cherry-picked off of #5030 #### Issues Ref https://linear.app/getsentry/issue/TET-1293/make-sure-that-agent-name-is-set-on-all-of-its-gen-ai-children #### Reminders - Please add tests to validate your changes, and lint your code using `tox -e linters`. - Add GH Issue ID _&_ Linear ID (if applicable) - PR title should use [conventional commit](https://develop.sentry.dev/engineering-practices/commit-messages/#type) style (`feat:`, `fix:`, `ref:`, `meta:`) - For external contributors: [CONTRIBUTING.md](https://github.com/getsentry/sentry-python/blob/master/CONTRIBUTING.md), [Sentry SDK development docs](https://develop.sentry.dev/sdk/), [Discord community](https://discord.gg/Ww9hbqr) --------- Co-authored-by: Fabian Schindler <[email protected]>

constantinius · 2025-11-04T13:23:02Z

I talked to the people at Relay and it seems to be feasible to put it in the place where the span-buffer is evaluated (not Relay itself).

So, I think this is a viable solution next to the ContextVar approach.

constantinius

Alright, this is now implemented using ContextVars.

sentry_sdk/integrations/langchain.py

…i-agent-name

sentry_sdk/integrations/langchain.py

sentrivana

Looks ok, but please double check the Cursor comments

…i-agent-name

fix(integrations): ensure that GEN_AI_AGENT_NAME is properly set for …

b86aea3

…GEN_AI spans under an invoke_agent span

constantinius requested a review from a team as a code owner October 24, 2025 13:44

constantinius requested a review from a team October 24, 2025 13:45

This comment was marked as outdated.

Sign in to view

sentrivana mentioned this pull request Oct 29, 2025

fix(google-genai): Set agent name #5038

Merged

Merge branch 'master' into constantinius/fix/integrations/ensure-gena…

5043e91

…i-agent-name

sentrivana reviewed Oct 29, 2025

View reviewed changes

constantinius mentioned this pull request Nov 7, 2025

feat(consumers): ensure that gen_ai spans get the gen_ai.agent.name from the ancestor span getsentry/sentry#102955

Merged

fix: using contextvar to store agent name in a re-entrant way

ea0a549

constantinius commented Nov 7, 2025

View reviewed changes

cursor bot reviewed Nov 7, 2025

View reviewed changes

sentry_sdk/integrations/langchain.py Outdated Show resolved Hide resolved

sentry_sdk/integrations/langchain.py Show resolved Hide resolved

Merge branch 'master' into constantinius/fix/integrations/ensure-gena…

f096f55

…i-agent-name

cursor bot reviewed Nov 13, 2025

View reviewed changes

sentry_sdk/integrations/langchain.py Outdated Show resolved Hide resolved

sentrivana approved these changes Nov 18, 2025

View reviewed changes

constantinius added 4 commits November 18, 2025 10:02

Merge branch 'master' into constantinius/fix/integrations/ensure-gena…

e9c0516

…i-agent-name

ref(langchain): Improve agent stack management for async contexts

012414a

fix(langchain): Enhance error handling in agent executor streams

a6ce3ac

test: Update token usage assertions to support newer langchain versions

b1d8fc0

constantinius merged commit 14aff96 into master Nov 18, 2025
131 of 132 checks passed

constantinius deleted the constantinius/fix/integrations/ensure-genai-agent-name branch November 18, 2025 12:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(integrations): ensure that GEN_AI_AGENT_NAME is properly set for GEN_AI spans under an invoke_agent span #5030

fix(integrations): ensure that GEN_AI_AGENT_NAME is properly set for GEN_AI spans under an invoke_agent span #5030

constantinius commented Oct 24, 2025

Uh oh!

linear bot commented Oct 24, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

codecov bot commented Oct 24, 2025 •

edited

Loading

Uh oh!

sentrivana left a comment •

edited

Loading

Uh oh!

constantinius commented Nov 4, 2025

Uh oh!

constantinius left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sentrivana left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fix(integrations): ensure that GEN_AI_AGENT_NAME is properly set for GEN_AI spans under an invoke_agent span #5030

fix(integrations): ensure that GEN_AI_AGENT_NAME is properly set for GEN_AI spans under an invoke_agent span #5030

Conversation

constantinius commented Oct 24, 2025

Description

Issues

Uh oh!

linear bot commented Oct 24, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

codecov bot commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 12 Tests Failed:

Uh oh!

sentrivana left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

constantinius commented Nov 4, 2025

Uh oh!

constantinius left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sentrivana left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Oct 24, 2025 •

edited

Loading

sentrivana left a comment •

edited

Loading