Skip to content

Adding a ThinkingBlock among content blocks #19919

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Sep 24, 2025
Merged

Adding a ThinkingBlock among content blocks #19919

merged 9 commits into from
Sep 24, 2025

Conversation

@AstraBert
Copy link
Member

@AstraBert AstraBert commented Sep 22, 2025 •
edited
Loading

Adding a thinking block and support for it (input/output) in:

  • Anthropic
  • Google GenAI
  • Mistral
  • OpenAI

Edit:

  • Bedrock apparently does not support thinking out of the box(?) -> Next PR
  • OpenAI had some versioning discrepancies, they are fixed now
  • Gemini did not support additional_kwargs merging in its message merging logic, this should be fixed as well

@AstraBert AstraBert marked this pull request as ready for review September 22, 2025 17:34
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Sep 22, 2025
@AstraBert AstraBert marked this pull request as draft September 22, 2025 20:06
@AstraBert AstraBert marked this pull request as ready for review September 23, 2025 11:54
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Sep 23, 2025
logan-markewich
logan-markewich approved these changes Sep 24, 2025
View reviewed changes
Copy link
Collaborator

@logan-markewich logan-markewich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems pretty good to me. I think two follow-up APIs would be

  • bedrock
  • ollama

I think in the future to, I might add a .content_delta param to the ChatResponse object, but thats also for another day 😁

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Sep 24, 2025
@AstraBert
Copy link
Member Author

@logan-markewich Just as a little DevX improvement, I added a check in Anthropic that automatically sets temperature to 1 if thinking is enabled :))

@AstraBert AstraBert merged commit 2965fc8 into main Sep 24, 2025
6 of 11 checks passed
@AstraBert AstraBert deleted the clelia/thinking-content-block branch September 24, 2025 15:43
frankiekim5 pushed a commit to frankiekim5/bedrock-agentcore-memory that referenced this pull request Sep 24, 2025
@AstraBert
Adding a ThinkingBlock among content blocks (run-llama#19919)
d881161 
* wip: add thinking block

* ci: lint

* feat: add/modify tests

* feat: add thinking block handling also as input

* fix: fixes for Gemini message merging; ci: tests tests tests

* chore: add support for ThinkingBlock also in OpenAI streaming; fix: DevX improvement in Anthropic by automatically setting temperature to 1 if thinking detected

* fix: add correct support for thinking in OpenAIResponses streaming

* fix: fix test and streaming

* chore: vbump (minor)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@logan-markewich logan-markewich logan-markewich approved these changes

Assignees

No one assigned

Labels

lgtm This PR has been approved by a maintainer size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@AstraBert @logan-markewich