Know what the caller heard: speech control for agents
October 25, 2026 · release
@voicethere/agent 0.11.0 · @voicethere/client 0.13.0
When a caller interrupts your agent, the agent used to know only that it had been interrupted. It could not tell whether the caller heard the first word or the whole answer, so an LLM kept a history in which the caller had heard everything.
speak() now returns a handle. Its done promise reports whether the reply was played, interrupted, cancelled or failed, and how many milliseconds played.
Barge-in also changed. When the caller talks over the agent, the reply that is playing stops and the interruptible speech still waiting in the queue is dropped, instead of starting right after the caller finishes. Anything sent with announce() or interruptible: false still plays in full, which is what you want for a legal notice or an order summary.
After a barge-in, onAgentInterrupted gives you playedText (what the caller heard) and unplayedText. The sentence that was cut is estimated from how much of its audio played, trimmed back to a whole word, and playedTextExact tells you when that estimate was used.
import { defineAgent, announce, speak } from "@voicethere/agent";
defineAgent({
async onSessionStart({ sessionId }) {
await announce(sessionId, "This call may be recorded.").done;
},
async onUserSpeechFinal({ sessionId }) {
const { status } = await speak(sessionId, "Your order ships Friday.").done;
if (status === "interrupted") console.log("caller cut in");
},
async onAgentInterrupted({ playedText }) {
console.log("caller heard:", playedText);
},
});
The browser client can show a "thinking" indicator with the new onAgentState option, fed by setAgentState in your agent. Runners that predate this return unsupported and keep playing speech as before.
Read the details in the speech control guide.