Outcome: In the roadside-assistance voice workflows we evaluated, Buffaly detected hallucinations and bad extracted data more than 80% of the time and prevented 100% of invalid tool calls from executing. The important part is not that the agent sounded better. It is that unsupported beliefs stopped becoming actions.
The clearest way to understand the problem is to watch one conversation go wrong. The caller says one thing. The voice agent builds a different internal story. Then the agent tries to continue the workflow as if that story were true.
The caller said Jonathan. The agent invented Paul Johnson.
The first objective was simple: collect the caller's first and last name before moving to membership verification. The agent did not have enough evidence to complete that objective, but it moved forward anyway.
Case 1: identity verification
A wrong name becomes the agent's working state
"Hello! My name is Sam, your roadside assistance agent. Could I start with your first and last name, please?"
"Hello?"
"Hello! My name is Sam, your roadside assistance agent. Could I start with your name, please?"
"Jonathan."
"Thank you, Paul. Could I have your last name as well, to confirm your membership?"
"Buh-bye!"
"Jonathan."
"Thank you, Paul Johnson. I see you've been a loyal member for a while now. For security, could you please provide your membership number?"
first_name = Paullast_name = Johnson
first_name may be Jonathanlast_name is unknown
Name collection is incomplete. Membership verification and downstream service tools stay blocked.
This is why the paper's method matters. Buffaly does not only ask the model, "What do you think happened?" It also asks, "What exact caller statement supports each fact?" That turns a vague hallucination complaint into a testable evidence problem.
The two checks
For every extracted fact, Buffaly asks two questions:
- Is the supporting text actually present in the caller's transcript?
- If it is present, does it actually support the extracted fact?
In the Jonathan call, both checks fail. "Paul" is not supported. "Johnson" is not in the caller transcript. The agent must clarify the name instead of advancing to membership verification.
The watermelon example shows two different errors
The second case is useful because it contains two failures that look similar if you only read the final transcript. One is a likely speech-recognition error. The other is a language-model hallucination.
Case 2: location confirmation
A transcription error gets repaired, but the hallucination remains
"Thanks, Paul. Now, could you tell me your current location, including the city and state? If you have a specific address or notable landmark nearby, that would be great for accuracy."
"I'm a watermelon."
"Got it, you're in Mims, Florida. Can you provide more details about your specific location?"
"Yeah, I'm at the Florida Mall."
"I'm a watermelon" does not satisfy a location objective. Buffaly keeps the objective open and can request retranscription or clarification.
"Mims, Florida" is not supported by either transcript. Retranscription improves the evidence, but it does not validate Mims.
The corrected transcript helps, but it does not finish the job. "Florida Mall" supports a landmark. It does not prove a city, state, service area, or dispatchable address. The location objective remains open.
Recovery point
The caller keeps trying to correct the agent
"Yeah, I'm at the Florida Mall."
"Got it, you're in Mims, Florida. Can you provide more details about your specific location?"
"I'm at the Florida Mall."
"Thank you for that information, Paul. Could you please describe the issue you're having with your vehicle?"
"I'm at the Florida Mall."
"I understand, Paul. Can you tell me what seems to be the problem with your vehicle?"
Correct the extracted information to Florida Mall and verify it with the caller before moving to the vehicle objective.
What OGAR changes
OGAR stands for Ontology Guided Augmented Retrieval. In this workflow, that means the conversation is represented as typed state: objectives, extracted facts, supporting statements, uncertainty, records, permitted tools, blocked tools, and recovery actions.
The language model can propose a fact. Buffaly decides whether that fact is supported. If the fact is unsupported, it is not written into trusted state. If a required objective is incomplete, tools that depend on that objective do not run.
-
Extract candidate facts
The model proposes values such as a caller name, landmark, city, or membership status.
-
Require supporting statements
Each proposed value must point back to the caller evidence or an authoritative record that supports it.
-
Validate the objective
The system checks whether the current workflow objective is actually complete.
-
Gate the next action
Clarification can stay open. Membership lookup, service-area selection, and dispatch stay blocked until their prerequisites are valid.
The business result
The practical result is measurable control over a risky voice workflow. In the tested data, Buffaly detected hallucinations and bad extracted data more than 80% of the time. More importantly, the typed action gate prevented 100% of invalid tool calls in the evaluated scenarios.
That second number matters because detection alone is not enough. A system can flag a hallucination after the fact and still let the agent call the wrong tool, verify the wrong person, choose the wrong service area, or dispatch to the wrong place. Buffaly's enforcement layer makes the invalid action unavailable until the required evidence exists.
This is the core lesson from the paper: reliable voice AI is not just a better transcript or a more careful prompt. It is a workflow that can prove what it knows, admit what it does not know, and refuse to act on unsupported state.
Want a review of your own voice-agent workflow?
Intelligence Factory can inspect a real workflow, identify where unsupported conclusions and invalid tool calls enter the process, and show where Buffaly would validate state, require clarification, or block unsafe actions.
Book a technical review of your AI voice workflow.