The Voice at the Counter

Chapter 1 · The problemThe Thesis

Voice interfaces do not fail on intelligence. They fail on timing. A model that answers perfectly in two seconds loses to a person who answers adequately in half of one.

Most voice products are built as a sequence of requests: record, upload, transcribe, think, speak. Every hop adds delay, and the conversation dies somewhere in the gaps.

Chapter 2 · The client & my roleThe Catalyst

Brewly set out to run a café as a live system: orders, status, and staff all moving in real time. The ambition was a customer who could simply talk to the counter.

I architected the foundation of the system and built the customer-facing Flutter application for web and mobile, then integrated a LiveKit voice agent that takes orders, assists customers, and updates the interface as the conversation happens.

Chapter 3 · The hard partThe Friction

The hard part was never the language model. It was the audio. A voice agent that speaks over people, pauses too long, or misses the end of a sentence feels broken, no matter how smart its answers are.

The decision was to treat the conversation as a stream, not a transaction. Audio moves over WebRTC through LiveKit, order state moves over WebSockets, and the agent drives the screen directly, so what the customer hears and what the customer sees never disagree.

“A correct answer delivered late is experienced as a wrong one.”

Chapter 4 · The resultThe Monument

Brewly runs on a single real-time spine. Orders and statuses propagate live across the café, and the voice agent is a first-class surface of the product rather than a bolt-on demo.

1

Flutter codebase serving web and mobile

Live

Order and status updates over WebSockets

3

Agent duties: ordering, assistance, live UI updates