Volume II · On-device AI · Voice agents
The Listening Machine
An always-on Android voice agent that waits for its name, then acts. And the two days spent discovering the model was never the problem.
- Problem
- An always-on Android voice agent whose wake word kept missing in real rooms.
- Outcome
- Wake word runs on-device with no licence fees, matching the commercial engine’s accuracy.
- Principal
- Tier75 · continued as independent research
- My role
- Voice systems engineer
- Period
- 2025 — Present
- Tech used
- Kotlin · Flutter · FastAPI · LiveKit · LangChain · LangGraph · openWakeWord · ONNX · AssemblyAI · ElevenLabs · Tavily
Chapter 1 · The problemThe Thesis
The phone is the last interface still waiting to be touched. An assistant that requires you to open it is a feature. An assistant that is already listening is a platform.
When an on-device model seems deaf, most teams retrain the model. They are usually wrong.
Chapter 2 · The client & my roleThe Catalyst
With Tier75, I helped build an end-to-end Android voice agent that runs continuously in the background, listens for a hotword, and executes tasks with tools: LangChain and LangGraph for reasoning, AssemblyAI and on-device recognition for speech, ElevenLabs and native TTS for voice, Tavily for live search, and Kotlin with WebRTC for the low-level work.
The work continues as a hands-free mobile agent: a Flutter interface, a FastAPI intelligence layer, LiveKit for real-time audio, and native Kotlin where Android audio performance is decided. It handles meeting summaries and email, without a single tap.
Chapter 3 · The hard partThe Friction
Commercial wake-word engines carried per-seat keys. Training our own on synthetic audio failed against real room noise and varied microphones. We moved to openWakeWord, which runs ONNX models locally and is fully open source, and detection was terrible.
We spent two days assuming the model was weak. It was not. The Linux development environment was feeding it mangled audio through PulseAudio and WirePlumber misconfigurations; the inference engine was evaluating corrupted buffers. Once the raw audio pipeline was fixed and the sample rate locked, accuracy matched the commercial engine, with no license attached.
“Most wake-word failures live in the audio daemon, not the inference layer. Audit the input before you touch the model.”
Chapter 4 · The resultThe Monument
The agent wakes on device, with no per-seat licensing and no cloud round-trip for the wake word. The lesson became doctrine: measure the input stream before believing anything about the system that consumes it.
- 0
- Per-seat wake-word licence keys
- On-device
- Hotword detection via ONNX
- 24/7
- Background listening service in Kotlin