- 
                Notifications
    
You must be signed in to change notification settings  - Fork 6.5k
 
Feat/bedrock converse partial system prompt cache #19922
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
          
     Merged
      
      
            logan-markewich
  merged 18 commits into
  run-llama:main
from
gringParis:feat/bedrock-converse-partial-system-prompt-cache
  
      
      
   
  Sep 29, 2025 
      
    
                
     Merged
            
            Feat/bedrock converse partial system prompt cache #19922
                    logan-markewich
  merged 18 commits into
  run-llama:main
from
gringParis:feat/bedrock-converse-partial-system-prompt-cache
  
      
      
   
  Sep 29, 2025 
              
            Conversation
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
    | 
           @logan-markewich @mattref  | 
    
              
                    mattref
  
              
              reviewed
              
                  
                    Sep 22, 2025 
                  
              
              
            
            
        
          
                llama-index-integrations/llms/llama-index-llms-bedrock-converse/pyproject.toml
              
                Outdated
          
            Show resolved
            Hide resolved
        
      
              
                    mattref
  
              
              reviewed
              
                  
                    Sep 22, 2025 
                  
              
              
            
            
        
          
                ...dex-integrations/llms/llama-index-llms-bedrock-converse/tests/test_bedrock_converse_utils.py
              
                Outdated
          
            Show resolved
            Hide resolved
        
              
          
                ...dex-integrations/llms/llama-index-llms-bedrock-converse/tests/test_bedrock_converse_utils.py
          
            Show resolved
            Hide resolved
        
      …ithub.com-personal:gringParis/llama_index into feat/bedrock-converse-partial-system-prompt-cache
              
                    logan-markewich
  
              
              approved these changes
              
                  
                    Sep 29, 2025 
                  
              
              
            
            
  
    Sign up for free
    to join this conversation on GitHub.
    Already have an account?
    Sign in to comment
  
      
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
Description
Allow system prompt caching with cache point mechanism as discussed here #19737 (comment)
I also noticed that using the system_prompt_caching parameter from that merged pull request raised an error by bedrock converse api. .
The system_prompt_caching parameter from BedrockConverse is preserved and can still be used.
Fixes # (issue)
fixes this issue
New Package? Its not a new package
Did I fill in the
tool.llamahubsection in thepyproject.tomland provide a detailed README.md for my new integration or package?Version Bump?
Did I bump the version in the
pyproject.tomlfile of the package I am updating? (Except for thellama-index-corepackage)Type of Change
Please delete options that are not relevant.
How Has This Been Tested?
Your pull-request will likely not be merged unless it is covered by some form of impactful unit testing.
Suggested Checklist:
uv run make format; uv run make lintto appease the lint gods