Is there a way to improve how often my agent uses its tool beyond the prompt?

[xuanyang15]

I'm seeing about a 44% success rate on using a tool when expected (it is always expected to use a tool).
The prompt is similar to this:

Search for products based on the user's query.
    Do not add any other text to your response than the json object.
    You must use the ... tool to search for products.
        
    1. For X product searches like "..." or "...", use the X tool. Return the products 
    in a json array like ....
    2. For Y searches like "...", think ... then search 
    ... using the X tool .... Return the products in a json array like ....

Should I give it examples of the tool call json it should generate?
Is there a way to train these models (gemini 2.0-flash or greater)?
Should I use memory in some way to make this better?
Should I simply cache results that I look up and bypass the high hallucination count?
Would provisioned compute help?

Some guidance I am considering: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/prompt-design-strategies#sample-prompt-template

[xuanyang15]

I think I've improved the response rate with the XML like design. TBD.

[xuanyang15]

Response from ADK Answering Agent

TLDR: Improving tool use reliability involves a combination of clear instructions, providing examples, using session state for context, and potentially fine-tuning for complex cases. Caching can help performance but won't fix the root issue of tool-use hallucination.

---

Hello! It's great that you're diving deep into improving your agent's performance. The issue of reliably getting an agent to use a tool is a common challenge. Here are some strategies based on ADK's design principles and best practices that can help, addressing your specific questions:

1. Should I give it examples of the tool call json it should generate?
   Yes, absolutely. Providing explicit examples in your prompt is a powerful technique. The LLM learns effectively from in-context examples (few-shot prompting). Clearly showing the model the exact format you expect is often more effective than just describing it. The user's XML-like design is a good step in this direction, as structured formats can help the model better understand the task.

2. Is there a way to train these models (gemini 1.5-flash or greater)?
   While ADK is model-agnostic and can work with various LLMs, including fine-tuned ones, fine-tuning is a more advanced step. [14] Before going down that path, it's highly recommended to exhaust prompt engineering techniques and other features ADK offers. If you consistently face issues with a specific, critical task, fine-tuning a model on a dataset of high-quality examples (prompt and expected tool call) can significantly improve reliability.

3. Should I use memory in some way to make this better?
   Yes, using memory, or more specifically Session State in ADK, can greatly improve performance. [1, 6] By storing context from previous turns in session.state, you provide the agent with more information to make better decisions. For instance, if a user's preference is stored in the state, the agent doesn't have to re-infer it, reducing the chances of error on subsequent tool calls. You can access this state within your tools via the ToolContext. [1, 11]

4. Should I simply cache results that I look up and bypass the high hallucination count?
   Caching is an excellent pattern for improving performance and reducing costs, but it addresses a different problem. [3, 9] Caching can prevent redundant tool calls for the same inputs, but it won't help the agent decide to use the tool correctly in the first place. If the agent fails to call the tool, the cache will never be hit. It's best to implement caching as a performance optimization after you've improved the tool-use reliability.

5. Would provisioned compute help?
   Provisioned compute primarily addresses latency and throughput, not the logical correctness of the agent's tool use. It ensures your agent runs in a consistent, low-latency environment but doesn't influence the LLM's reasoning process for tool calling. It's a scaling and performance optimization, not a solution for hallucination.

Additional Recommendations from ADK Best Practices:

• Clear Tool Docstrings: The LLM relies heavily on the tool's function name, parameters, type hints, and docstring to understand what it does. [11] Ensure your tool's docstring is descriptive and clearly explains its purpose, arguments, and what it returns.
• Guardrails with Callbacks: For more robust control, you can use before_tool_callback. This allows you to inspect and even modify the arguments the LLM generated before the tool is executed, or to block the call entirely if it's invalid. [3]

By combining these strategies—especially more detailed prompting with examples and leveraging session state—you should see a significant improvement in your agent's tool-use success rate.

[1] https://github.com/google/adk-docs/blob/main/docs/tutorials/agent-team.md
[3] https://github.com/google/adk-docs/blob/main/docs/callbacks/design-patterns-and-best-practices.md
[6] https://github.com/google/adk-docs/blob/main/docs/sessions/index.md
[9] https://github.com/google/adk-docs/blob/main/docs/artifacts/index.md
[11] https://github.com/google/adk-docs/blob/main/docs/tools/index.md
[14] https://github.com/google/adk-docs/blob/main/docs/agents/models.md