Skip to content
All articles
AI Engineering

How to Reduce Hallucinations in AI Chatbots

Why chatbots invent facts, the grounding ladder from prompt rules to code-decided answers, and the production refund agent that resolves a user's real state before the model writes a single word.

Rushit Jivani6 min read

The short version

  • Hallucination is not a bug you patch with a better prompt — generation is guessing by design. The fix is architectural: control what the model knows, and remove it from decisions entirely where the stakes are high.
  • Climb the grounding ladder per answer type: prompt rules, then retrieved and live context, then facts decided in code before generation, then no generation at all for the riskiest replies.
  • The case study: a refund agent that resolves one of four account states in code — signed out, several events, no ticket, one ticket with the real refund flag — and in the most dangerous state answers with a code-built list, no model call at all.

Every team that ships a chatbot meets the same failure within a week: the bot states something false, fluently, with total confidence. A refund policy that does not exist. A deadline nobody set. A feature from some other product. Users call it lying; researchers call it hallucination; mechanically it is neither — it is a text generator doing exactly what it was built to do, which is produce plausible continuations whether or not the facts behind them exist.

That framing matters because it tells you where the fix lives. You cannot prompt a guesser into being a database. You can, however, change what it knows, narrow what it is asked to decide, and remove it entirely from the answers that matter most. This guide walks that ladder, then shows the rung-by-rung implementation in a production refund agent — the highest-stakes path of a support bot on a live events platform.

Why models hallucinate

A language model answers by predicting likely next words given the context. If the true answer is in the context, the likely continuation is usually that answer. If it is not, the model does not look anything up — it has nothing to look things up in — so the likely continuation is whatever typically follows such a question: a plausible policy, a reasonable-sounding deadline. Fluency and truth come from the same mechanism, which is why confident nonsense is the default failure, not a rare glitch.

Three design consequences follow:

  1. Missing context is an invitation. Any fact the answer needs that the prompt does not contain will be guessed.
  2. Instructions bound style more than truth. "Do not make things up" reduces reckless phrasing; it does not create knowledge.
  3. The only reliable fix is structural. Supply the facts, shrink the decision, or remove the generator from the loop.

How you should reduce it

Work the ladder from bottom to top, and spend higher-rung effort on your riskiest answers first:

The grounding ladder

  1. 1“Do not hallucinate” in the promptNecessary wording, but the model is still the one deciding what is true. Never the only control.
  2. 2Ground the prompt — RAG + live contextPut the actual policy text and the actual record in the context, and instruct the model to stay inside them.
  3. 3Decide facts in code before the model runsLook up the account, the record, the flag. Branch on the result. The model writes sentences for a state that is already true.
  4. 4Remove the model from high-stakes repliesBuild the risky reply in code — a list, a verdict, an authored fallback. Zero generation means zero invention.
Climb the ladder for your riskiest answers first. A refund verdict deserves rung 3 or 4. A how-do-I-publish answer is fine on rung 2.

Rung 1: prompt rules — necessary, never sufficient

Write the refusals explicitly: answer only from the provided material; if the answer is not present, say so and point to support; never invent a policy, price, deadline, link, or feature. Forbid the specific inventions your domain fears — a named list beats a vague "be accurate." This rung costs minutes and catches the reckless phrasings. It decides nothing about truth.

Rung 2: ground the prompt

Give the model the actual facts: retrieved knowledge-base passages for the stable rules (the RAG guide covers the pipeline), and live request data — the page the user is on, their role, current record values — pasted in as data. Add a precedence rule for conflicts (live record beats stored prose). Most everyday hallucinations die here, because the likely continuation of a prompt containing the true policy is the true policy.

Rung 3: decide facts in code before the model runs

Rung 2 still lets the model interpret facts. For consequential answers, stop asking it to. Look up the record in code, branch on the result, and give each branch its own prompt that states the already-decided facts: "This user has no active order — tell them that." The model's job collapses from "figure out what is true" to "phrase this truth well." Hallucinating is still possible; being wrong about the state is not, because the state was never its call.

This rung has a corollary people miss: clarifying is a verdict too. When the facts are ambiguous — the user could mean two different orders — the grounded answer is a question, built from the records, not a guess about which one they meant.

Rung 4: remove the model from the riskiest replies

Some outputs are too consequential for any generation: verdicts about money, lists of a user's own records, anything a user will act on irreversibly. Build those replies in code — a formatted list, a fixed paragraph with the real link. And write authored fallbacks for every model-dependent path, so provider failure degrades to accurate static text. Zero generation is the only zero hallucination.

One more honesty rule for the whole ladder: if you have not measured hallucination rates with an eval set, do not claim percentages. Say which structural controls exist. Structure is verifiable; vibes are not.

How I implemented it on a real project

The sharpest hallucination risk on the events platform I work on is the refund conversation. The nightmare replies are concrete: telling someone they can leave an event they hold no ticket for, or promising a refund the organizer turned off. Both are "invented facts about records" — rung 3 and 4 territory. So the refund agent was built as a state machine that resolves what is true before any model is invoked. Walk the four states:

Four states, decided before the model speaks

A numbered list — no model callpure code — nothing to hallucinate

The account holds tickets to more than one upcoming event and the question names none of them. The reply is built in code: each event with its link, ticket count, and date, plus one question — which one?

The state machine runs in code. The model only writes sentences for a state that was already chosen — and one state skips it entirely.

State 1 — not signed in. No identity, so no ticket lookup happens at all. The prompt may explain organizer-set policies from FAQs, must include the real sign-in link (supplied by code, not recalled by the model), and must not name an event they "probably" bought. The fallback is a fixed paragraph with that same link.

State 2 — several upcoming events, none named. The dangerous one. The account holds tickets to multiple events and the question names none. The reply is not a model call. Code builds a numbered list — each event with its link, ticket count, and date — and asks which one they mean. There is no opportunity for the model to drop an event, merge two names, or add a third, because no generation occurs. Asking the clarifying question is the product; answering anyway would be the bug.

State 3 — signed in, no ticket. The prompt states that the account was checked and nothing active exists. It may suggest a different email; it points at the support address; it is forbidden from offering leave-event steps for a ticket that is not there. If the record load itself fails, the code logs it and treats the list as empty — degrading to this cautious state rather than guessing an order into existence.

State 4 — one ticket, real flag read. Only now is the event's refund flag read, and the prompt splits on its actual value. Refunds off: say the organizer made the event non-refundable, point to Contact Host, and — written into the prompt because models try it — never promise the host will make an exception. Refunds open: give the product's real steps, and never invent a processing time. Each branch has its own authored fallback, so an outage cannot flip a no-refund event into a refund.

Around the state machine sit the other rungs: retrieval biased toward policy FAQs (rung 2), the no-inventing rules (rung 1), and a human-takeover short-circuit that lets a person own any thread (that system is here). There is no eval-set percentage to report, and I will not invent one — the claim is the architecture: the branch happens in code, the forbidden promises are written down, and the failure text is authored by us.

Pitfalls to avoid

  • Treating the system prompt as the whole strategy. Rung 1 alone leaves the model deciding whether a record exists.
  • Letting the model resolve ambiguity about records. Ambiguity detection belongs in code; the model should receive either one resolved fact or the instruction to ask.
  • Grounding the happy path only. The error paths — lookup failed, provider down — are where guessing sneaks back in. Author those texts.
  • Shipping confidence theater. An invented "95% accuracy" is itself a hallucination. Report controls, not imagined metrics.

FAQ

No. It makes output deterministic, not true — the most likely guess is still a guess. Low temperature helps format-bound tasks like classification; grounding is what bounds truth.

  • #Hallucinations
  • #AIChatbots
  • #Grounding
  • #Guide
  • #CaseStudy
Rushit Jivani headshot

Written by

Rushit Jivani

Senior Software Architect & AI Application Engineer. Architecture that shows up in crash rate, performance and how fast the team ships.