Why website chatbots often make things up, and how to stop it
Language models are trained to always have an answer, even when they don’t know. This is how retrieval and guardrails work to prevent that.
The most common complaint about chatbots on business websites is always similar. The bot answers confidently, with neat sentences, but the content is wrong. Some state prices that don’t exist. Others promise guarantees that aren’t offered. Customers trust the official-sounding answers, then the sales team has to fix it.
The cause isn’t a broken bot. Language models are designed to construct the most likely sentence following a question, not to check if the content is true. If it doesn’t have the information, it still makes something up. Technically, this is called hallucination, and it’s inherent, not accidental.
The difference between a basic chatbot and a content-equipped chatbot
A basic chatbot simply forwards visitor questions to the model and displays the response. The model answers from its training data, which never includes your site’s content. So anything specific to your business—from prices to hours—is made up.
InWebChatBot takes a different approach in one key step. Before the model answers, the system first searches your site’s content for the closest match to the question, then inserts that snippet into the prompt. The model isn’t asked to remember; it’s asked to summarize existing material. This pattern is called retrieval-augmented generation.
- 1Site content is taken via the sitemap, cleaned of navigation and footers, then broken into small pieces.
- 2Each piece is converted into a vector—a kind of meaning fingerprint—and stored in a database.
- 3When a visitor asks a question, the question is also converted into a vector, then the closest matches are found.
- 4Selected snippets are sent to the model along with your question, with instructions to answer only from that material.
Retrieval alone isn’t enough
Retrieving relevant content solves most problems, but leaves one gap. Visitors may ask about things your site doesn’t answer. Without safeguards, the model will fill that gap with plausible guesses.
That’s where a guardrail works. A guardrail is a rejection statement you write yourself. The bot uses it when it doesn’t have enough information to answer. The bot doesn’t stay silent or make things up. It says it doesn’t know, then directs to a path you define.
Sorry, that information isn’t yet on our pages. For specific needs like this, please contact our team via WhatsApp at the number on our Contact page.
A sentence like the one above is much safer than a fabricated answer, and often more helpful. Visitors get a next step, not false information.
Three common reasons for incorrect answers
The source content doesn’t exist
No system can answer price questions if the price isn’t written on the site. Before blaming the bot, open the page that *should* contain the answer and make sure it’s actually there in text form.
The page isn’t being read
If your sitemap doesn’t include a certain page, its content never enters the knowledge base. Sites with content that only appears after JavaScript runs are also prone to being read as empty. This is the most common and often overlooked source of problems.
The content has changed but hasn’t been refreshed
The knowledge base is a copy of your site’s content at the last extraction. Change a price on a page without triggering a refresh, and the bot will continue to answer with the old price for days.
Metrics to monitor
After the bot is running, the most useful thing to read isn’t the number of messages, but the content of the conversations. Chat history shows what questions visitors are actually asking. Repeated questions answered with a guardrail are a clear to-do list: that’s content missing from your site and worth writing.
A good chatbot ultimately improves its own site, indirectly. It highlights gaps in content that were previously invisible from analytics.