10 Seconds to Zero: Rebuilding My Translator on Gemini's Streaming API
The old version of TransLee worked. People used it. It just made you wait. The Old Pipeline: Record, Wait, Stitch The original app was built in Flutter, with a Node.js server in the middle talking to OpenAI’s Whisper for speech-to-text and GPT for translation. The flow looked like this: [Record 5-15 seconds] -> [Whisper: speech to text] -> [GPT: translate text, using the tail of the previous chunk for context] -> [Text-to-speech] -> [Play the translated audio] ⬑ repeat, waiting the full cycle each time ⬏ Five to fifteen seconds isn’t a random number. It was a setting I chose on purpose, because shorter chunks meant the translation kept losing the thread mid-sentence, and longer chunks meant a longer wait before you heard anything at all. The fix for the “losing the thread” problem was to keep the last bit of the previous chunk’s text and glue it onto the front of the next one before sending it off to be translated, so GPT had enough context to know what was going on. It worked. It also meant every single sentence arrived several seconds after it was actually spoken. ...