Audio transcription returns mixed AI response or no text (iOS & Android)

[jin-reyapp]

Key Symptoms Observed

• Transcription frequently returns a mix of transcribed text and an AI-generated response instead of only the transcribed audio.
• The mic icon shows a loading state but returns no text in the input field.
• Sometimes the mic icon stays loading for 1-2minutes before any transcription is returned.

Step to Reproduce

1. Open the app on iOS(TestFlight) or Android.
2. Tap the mic icon to start audio transcription.
3. Speak into the mic for a few seconds.
4. Observe the input field behavior after the loading state.

Environment

• iOS (TestFlight)
• Android
• Flutter AI Toolkit v0.8

ScreenRecording_05-05-2025.3-00-16.PM_1.2.MP4.mp4

[csells]

@jin-reyapp can you tell me what text you recorded? there's no audio in the screen recording. thanks.

also, can you post a minimal repro project? I'm curious what model you're using that shows this problem.

[miquelbeltran]

Hi, like mentioned in #95 (comment), I think this is more a problem on the nature of LLMs than a coding issue in this project.

I can see similar issues when recording audio but remaining silent, but then the transcription just generates some text based on the LLM conversation history and system configuration.

For the time being, we have disabled the audio transcription feature in our app, as it was unreliable.

[csells]

I think with a little bit of explicit prompt engineering, we can reduce this quite a bit. I'll take a crack at it.

[csells]

@miquelbeltran I've implemented a branch that provides a hook in LlmChatView to allows you to override the audio to text translation mechanism after the audio has been recorded. Check out the custom_sst sample for how to use it. I happen to use gemini 2.5 flash as a known good stt translator in the sample because I have it handy, but you could use native libraries instead to do the translation.