The TWIML AI Podcast

The TWIML AI Podcast ·Sam Charrington and Boson AI co-founder Alex Smola discuss why natural voice interaction is a systems problem rather than simply a more capable language model. Charrington describes brittle experiences with current voice interfaces; Smola counters with a noisy-bar demonstration while crediting phone-side audio processing for part of its success. Smola connects interruption latency, audio tokenization, model size, streaming buffers and audiovisual consistency to a central design constraint: a sustained conversation must be affordable. He describes working backward from a price target instead of maximizing model scale first. Smola traces Boson's move from text toward audio, distinguishing speech recognition from audio understanding and emphasizing the risk of losing language capabilities when adding a modality. He describes a large audio collection and processing effort, privately operated infrastructure, weak supervision and an iterative model-annotation loop. In his account, adaptation involves substantial training rather than light fine-tuning. He then explains how foreground conversation can coexist with background reasoning and tool calls, with context-window and KV-cache costs limiting integration choices. His claimed benchmark advantage comes with a material qualification: the comparison uses competing models with thinking turned off. The later discussion shifts from task completion to the quality of human interaction. Smola argues for communicating waits, distinguishing backchannels from interruptions, recovering gracefully and noticing unstated needs. He names ProactBench and IHBench, cautions against treating fictional or entertaining interactions as normal behavior, and preserves an ethical boundary around experiments involving people. The closing discussion separates simulated-user training, personal memory, population-level improvements and cultural adaptation. Smola suggests that new interactions and explicit instruction could teach rare communication skills, while framing rapid progress over the next two or three years as an expectation rather than an established outcome.