How to get Agent responses not go to LLM?

[shivsant]

I am not sure if it's possible.
I am reading a large file through a read_file tool in my agent.
There are multiple tools available to the agent - like read_file, write_file etc. So I want the LLM to make the reasoning/decision to call the appropriate tool i.e. read the file in this case.

But I don't want the LLM to read the file content. If it reads, sometimes the content in truncated or if the full file is available, it's going to be costly.

Is there a way to not send the output of a tool to LLM?

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: Yes, you can prevent a tool's output from being sent to the LLM by using an after_tool_callback to intercept and modify the tool's result before it is passed back to the model. You can store the actual output in the session state if you need to access it later.

---

That's an excellent question. Handling large tool outputs is a common challenge, and you are right to be concerned about the cost and potential for truncation. The Agent Development Kit (ADK) provides a flexible mechanism to achieve exactly what you're asking for through callbacks.

The recommended approach is to use the after_tool_callback. This callback function is executed after your tool runs but before its output is sent back to the LLM for the next reasoning step. Within this callback, you can inspect, modify, or completely replace the tool's result.

Here’s a conceptual example of how you could implement this:

1. Modify Your Tool to Use Session State: Instead of returning the large file content directly, your read_file tool can store the content in the session state. The session state is a dictionary (session.state) that persists across conversational turns within a single session and is accessible to tools and callbacks. [1]

   from google.adk.tools import ToolContext
   
   def read_file(file_path: str, tool_context: ToolContext) -> str:
       """Reads a file and stores its content in the session state."""
       try:
           with open(file_path, 'r') as f:
               content = f.read()
           # Store the large content in the session state
           tool_context.state['last_file_content'] = content
           return f"Successfully read and stored content from {file_path}."
       except Exception as e:
           return f"Error reading file: {e}"

2. Implement an after_tool_callback: You can define a callback function that checks if the tool that just ran was your read_file tool. If so, it can replace the output with a more manageable message, or even an empty dictionary, before it goes to the LLM.

   from google.adk.runtime import AfterToolCallbackContext
   
   def after_tool_interceptor(context: AfterToolCallbackContext) -> dict:
       """Checks for read_file tool output and prevents it from going to the LLM."""
       # Check if the tool that just ran is the one we want to intercept
       if context.tool.name == 'read_file':
           # The tool's actual return value is in context.result
           print(f"Intercepting result from {context.tool.name}: {context.result}")
           # Replace the result with a simple confirmation message.
           # This is what the LLM will see.
           return {"status": "File content was read successfully and is stored."}
       
       # For all other tools, return the original result
       return context.result

By using this pattern, the LLM is still responsible for deciding to call the read_file tool, but it never receives the large, costly output. The actual file content is available in session.state['last_file_content'] for other tools or parts of your application to use as needed.

This approach gives you fine-grained control over the data flow in your agent, allowing you to optimize for performance and cost while still leveraging the LLM's reasoning capabilities.

For more details, you can refer to the ADK documentation on Callbacks and Session State. [1, 3]

[1] https://google.github.io/adk-docs/tutorials/agent-team/
[2] https://google.github.io/adk-docs/callbacks/
[3] https://google.github.io/adk-docs/sessions/state/

[boyangsvl]

Closing as a duplicate question with #2619