Automatic Language switching in voice ai agents

[saurabh-emvo]

Is there a way to build voice ai agents with option to execute multiple language switching during the same conversation without the native audio models since they don't support RAG and tools right now?

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: It is possible to build voice AI agents that support both RAG and tools using the ADK. The limitation you mentioned regarding native audio models is not present in the current version of the ADK.

---

Hello! I can provide some information on this.

The ADK framework is designed to support a rich ecosystem of tools, including Retrieval-Augmented Generation (RAG), for building advanced voice AI agents. This functionality is available even when using models that support streaming audio.

RAG and Tool Support

The ADK has a specific tool called VertexAiRagRetrieval which enables your agent to retrieve information from a RAG corpus. [1] This works seamlessly with voice-enabled agents, allowing for dynamic and data-driven conversations.

Language Configuration

Regarding multi-language support, you can configure the language for your agent's speech-to-text and text-to-speech functionalities within the RunConfig object by setting the language_code in the speech_config. [2] While the documentation doesn't explicitly detail a mechanism for automatic language switching within a single, continuous conversation, the framework is flexible, and you can manage the language_code as part of your application's state.

For more in-depth information, you can refer to the following resources:

• About the ADK
• ADK Tools Overview
• Streaming in ADK
• RunConfig Options
• Vertex AI RAG Retrieval Tool Source Code

I hope this helps you build your voice AI agent!

---

Footnotes
[1] https://github.com/google/adk-python/blob/main/src/google/adk/tools/retrieval/vertex_ai_rag_retrieval.py
[2] https://google.github.io/adk-docs/runtime/runconfig/

[hangfei]

Tools(RAG etc) are supported. You can set a lanaguage for all the agents but not switching them among multiple agents.
(technically it's possible so you can raise a feature request. )