We built and shipped a voice solution for a national roadside-assistance program. The two conversations below came out of that work, and both are real. They show a failure mode that a fluent voice hides very well: the agent's internal picture of the call quietly stops matching what the caller actually said.
A voice agent that sounds competent is easy to trust, and that is the trap. What comes out of the speaker is generated text, and generated text is under no obligation to match the audio that went in. Somewhere between the microphone and the tow truck, something has to decide what the system actually believes and what it is allowed to do about it. That is the part we build.
Buffaly keeps an internal ontology of the call that grows turn by turn. It holds the caller, the facts, the evidence behind each fact, what is still uncertain, the objectives still open, the tools on hand, the actions already taken, and the actions currently available. Everything below is that model doing its job on two calls that went wrong in two different places.
Turn one: the caller said Jonathan
The provider's name is redacted. Everything else is presented as supplied.
What was said
"Hello! My name is Sam, your [REDACTED] roadside assistance agent. Could I start with your first and last name, please?"
"Hello?"
"Hello! My name is Sam, your [REDACTED] roadside assistance agent. Could I start with your name, please?"
"Jonathan."
"Thank you, Paul. Could I have your last name as well, to confirm your membership?"
"Buh-bye!"
"Jonathan."
"Thank you, Paul Johnson. I see you've been a loyal [REDACTED] member for a while now. For security, could you please provide your [REDACTED] membership number?"
What Buffaly sees
The caller said "Jonathan" twice.
first_name = Paul contradicts direct caller evidence.
last_name = Johnson has no supporting evidence.
No authoritative membership lookup supports the claimed history.
Identity remains unresolved. Paul and Johnson are not written into trusted state.
Membership verification and service actions.
Ask the caller to repeat or spell the name.
Read only the agent's side and it sounds like a professional handling a call. Read both sides and you watch a small error turn into a large one. In four turns the agent produced a first name that contradicts the caller, a last name nobody ever said, and a sentence about membership history that reads like it came from a record lookup that never happened.
Look at how the errors stack. Once Paul is in state, asking for a last name is a reasonable next move. Once Paul Johnson is in state, greeting a loyal member is reasonable. Once membership is implied, asking for the membership number is reasonable. Every step is locally sensible and the whole thing is wrong. That is why "the agent hallucinated" is a useless bug report. The question worth answering is which turn was the first one where the backend state stopped matching the evidence, and why the system kept granting permissions after that point.
Buffaly's handling of this call is deliberately boring. "Jonathan" enters as evidence, tied to the turn it came from. The model then proposes first_name = Paul, and that candidate dies on arrival because it contradicts what the caller said. last_name = Johnson dies because nothing supports it. The membership history claim needs an authoritative record behind it, and no lookup ran, so it never becomes trusted state either.
What remains is an identity objective that is still open, and that single unresolved fact is what closes the rest of the call. Membership verification and every service action stay blocked, because they all depend on knowing who is on the phone. The one thing that opens up is asking the caller to repeat or spell the name. The agent is not punished for being wrong. It just does not get the keys to anything that requires being right.
Further down the script: one turn, two different failures
The second conversation gets past identity and into location, then fails twice inside a single turn. The two failures look similar in a transcript review and need completely different repairs.
What was said
"Thanks, Paul. Now, could you tell me your current location, including the city and state? If you have a specific address or notable landmark nearby, that would be great for accuracy."
"I'm a watermelon."
"Got it, you're in Mims, Florida. Can you provide more details about your specific location? Or any nearby landmarks to help us."
"Yeah, I'm at the Florida Mall."
What Buffaly sees
"I'm a watermelon" does not satisfy the open location objective. Buffaly keeps the location unresolved and allows retranscription or clarification.
"I'm at the Florida Mall" supports caller_landmark = Florida Mall, but the location is still geographically ambiguous.
caller_city = Mims has no evidence. Mims is not written into trusted state.
Service-area selection and dispatch.
Confirm which Florida Mall the caller means, retrieve an authoritative address, or ask for another landmark.
"I'm a watermelon" is a speech recognition problem. The caller said words and the recognizer wrote down different words. The turn was supposed to satisfy an open location objective, a watermelon does not satisfy it, so the location stays unresolved and the recovery paths stay open.
"Mims, Florida" is a different animal. No recognizer produced Mims. The language model produced Mims to fill a slot the workflow wanted filled, then said it back to the caller with complete confidence.
That distinction decides the fix, and the two fixes are not interchangeable. Retranscription repairs recognition evidence. Run the audio again and "I'm a watermelon" becomes "Yeah, I'm at the Florida Mall," which enters the model as evidence and moves the location objective forward. Retranscription cannot validate Mims, Florida. There is no audio anywhere in that call that supports Mims, so no rerun will ever produce support for it. It fails validation before the correction and it fails after.
The corrected evidence does not close the objective either. Florida Mall supports caller_landmark = Florida Mall. It does not give you a dispatchable location. Service-area selection and dispatch stay blocked. What opens up is confirming which Florida Mall the caller means, retrieving an authoritative address, or asking for another landmark. Better evidence moved the call forward one step. It did not finish the step.
What is actually running underneath
Both calls go through the same loop on every turn. They fail at different points in it, which is exactly why they get different recovery options.
-
Capture the evidence
Audio, transcription, speaker, and time enter the conversation as observations. The transcript is evidence about the call, not the truth of the call.
-
Propose candidate facts
The language model reads the turn and proposes. Jonathan may be a first name. Florida Mall may be a landmark. Nothing it proposes is believed yet.
-
Update the internal ontology
Buffaly maintains a live model of the caller, facts and the evidence behind them, uncertainty, open objectives, authoritative records, tools, prior actions, and the actions currently available.
-
Validate the state
Candidate facts and claims are checked against transcript evidence, records, and what the workflow requires. Contradicted or unsupported values do not become trusted state.
-
Recalculate what is allowed
Identity unresolved means membership and service actions stay closed. Location unresolved means dispatch stays closed. Clarification and authoritative lookups open up instead.
-
Write back the validated result
The clarification, retranscription, lookup, correction, or escalation becomes the next event in the ontology, carrying the evidence and validation trace that produced it.
The source of truth grows with the conversation, but only through validated updates. One bad transcription or one invented city cannot quietly become the foundation for everything that follows.
The model decides what can happen next
This is the operational difference, and it is worth being blunt about. Plenty of systems can flag a sentence as probably wrong after the fact. Flagging does not stop a truck from being sent. Buffaly uses the validated model of the conversation to decide which actions are available in the first place, so an unsupported fact never becomes an authorization.
In the name call, the identity objective stays open. The agent can clarify or ask for a spelling. It cannot treat Paul Johnson as a verified person and it cannot move into membership-dependent service work.
In the location call, retranscription supplies better evidence, but Florida Mall still has to resolve to an authoritative location. The agent can ask which mall the caller means or pull an address. It cannot pick a service area or dispatch assistance on the strength of Mims.
Recovery is part of the workflow, not an exception to it
Every validation failure has a defined response. Ambiguous audio can trigger retranscription or a clarifying question. An unsupported value gets dropped from working state. A claim that depends on a record has to wait for an authoritative lookup. An action stays blocked until its evidence and policy checks pass. Contradictory evidence or a request outside scope gets escalated with the trace attached.
None of that is error handling bolted on the side. It is the same loop running with a different set of available actions, which is why a call that goes sideways on turn four can still be recovered on turn five instead of quietly continuing on bad state.
Being right and finishing the job are separate questions
The same model tracks whether the call ever completed its actual objective. An agent can avoid every false statement and still fail the caller by skipping identity, skipping authorization, missing required details, never taking the service action, or never confirming the problem was solved. A warm closing line is not evidence that the work is done, and a transcript full of polite sentences is not a completed workflow. Open objectives are state, so an unfinished call shows up as an unfinished call.
The trace is what makes it fixable
After a call, the team can reconstruct what was said, which candidate facts were proposed, what evidence supported each one, which checks passed or failed, which actions were blocked, what recovery followed, and whether the conversation reached its objective.
That turns "the agent hallucinated" into a specific engineering problem with an owner. The root cause might be transcription, extraction, missing evidence, a rule that is wrong, a tool that failed, or a workflow definition with a hole in it. Those are six different fixes, and without the trace you are guessing which one you need.
What reliable voice AI actually takes
Sounding natural is table stakes now, and it is the easy half. The hard half is an agent that separates evidence from assumption, keeps uncertainty instead of resolving it early, follows the real process, uses authoritative records, stops when an action is not permitted, recovers when something breaks, and can explain afterward what it did.
That is the layer we build. Buffaly connects what the caller said to typed knowledge, validation rules, tools, policy checks, and executable workflow, so the agent runs inside a system built to hold up under review rather than one built to sound good in a demo.
Want a look at your own voice-agent workflow?
Intelligence Factory can review a real voice-agent workflow and identify where unsupported conclusions, transcription errors, missing evidence, and unfinished objectives enter the process. We can then show how Buffaly validates the state, drives the correction, and enforces the operating rules that matter to your business.
Book a technical review of your AI voice workflow.