unsaid
for the words that got cut off
A voice agent that hears what was almost said. Turns that were cut off or abandoned are recorded by every transcription tool as finished turns. This one reports them as they happen, and when somebody loses the floor it waits for the room to settle and offers it back.
precision on 200 blind hand-labelled turns, at the threshold that ships.
recall on the same turns. It misses about half of what is there.
the separate, higher bar before the agent says anything out loud.
What it does
It listens to a multi-speaker call on AssemblyAI's streaming API, one socket per participant, and reports two kinds of turn that did not survive. Disrupted: somebody started talking within 0.3 seconds and the thought died there. Abandoned: the speaker stopped mid-thought and never came back to it. A third case, where somebody pauses and then picks the same thought up again, is deliberately not reported, because nothing was lost.
Every finding carries its evidence in plain words, such as “ends inside a word ('s'), C already talking”. When the agent acts, it quotes the words that died and asks: “B started to say something a moment ago and did not finish: ‘you don't want it plugged in really, s’. B, did you want to come back to that?”
Key decisions
It waits for the room to settle
The agent could speak the moment it notices a lost turn. It does not. It waits for real silence and drops the intervention if the silence never comes, because a tool whose whole subject is people being talked over cannot talk over people. A missed intervention costs a moment of silence; a false one costs the room's trust, after which nobody hears the correct ones either. It also never raises a thought the speaker recovered on their own, and lets old ones go, since an objection from eleven minutes ago is no longer a live thread.
Two thresholds, on purpose
The detector ships at 0.6, which suits a transcript read afterwards. The voice in the room needs 0.75. A thing you read later and a thing said out loud are different decisions, so they get different numbers. Turning the agent off with --no-floor changes no finding, which keeps the published precision and recall a description of what actually runs.
The turn threshold is a dial, not a fact
AssemblyAI exposes its turn detection settings, and the turns that fall between two settings are the product. The default model would have deleted it: it formats every turn, ends it with a full stop and repairs the broken last word, which is the clearest signal there is. So the stream is pinned to the model that leaves a turn as it was spoken, and the API's own end-of-turn confidence is left unused, because the corpus has no equivalent and a detector leaning on it could never be scored.
No language model
The sentence it speaks is a template. An agent trusted in a room cannot paraphrase what somebody meant, because the one error nobody forgives is being told what you were about to say.
Results
The AMI meeting corpus has human annotation, but its annotators marked disfluency and dialogue acts; nobody asked them which turns failed to finish. A turn ending on “the” carries an annotation only 29% of the time. So there is a second ground truth: 200 turns labelled by hand through a mechanical blind, with whether the detector flagged each one kept in a separate key file until the labels were written.
| Threshold | Precision | Recall | F1 |
|---|---|---|---|
| 0.50 | 0.332 | 0.482 | 0.393 |
| 0.60 (ships) | 0.737 | 0.482 | 0.582 |
| 0.70 | 0.757 | 0.456 | 0.569 |
Raising the threshold from 0.5 to 0.6 more than doubles precision for no loss of recall; raising it further buys little precision and starts losing real findings. Against the AMI annotations across 139 meetings the detector scores precision 0.665 and recall 0.277, lower because many turns it finds are real and unannotated.
The work found its own bugs. The dictionary listed headwords and not their forms, so a sentence ending on “hills” read as a word cut off midway; every number was re-measured after the fix rather than patched. And the loop that lets the agent speak during a pause had been written and never started, so live it would have stayed silent until the call ended while every test stayed green. There are 107 tests, and a sibling mutation-testing tool found six holes the suite was hiding.
Where it stands
Recall at 0.48 is the weak number. I wrote the detector and I wrote the labels; the blinding is mechanical, but a second labeller would be worth more than anything I can say about my own care. Built for the AssemblyAI Voice Agent Hackathon in September 2026.
The replay plays real AMI meetings with the agent speaking at the moments it chooses, and try it live needs only a microphone.
Meeting audio and annotation from the AMI Meeting Corpus, CC BY 4.0. The figure shows meeting ES2007b from 7:50 to 9:12 with the detector's findings exactly as the replay page ships them.