AI chatbot
By Saurav | Founder of saavos | Building in public toward $10k MRR
[!TLDR] When your chatbot hits a question outside its training data, it has three options: hallucinate an answer, say "I don't know" and dead-end the visitor, or route them somewhere useful. Most out-of-the-box setups do the first two. A properly configured fallback does the third. This post covers exactly what to configure, what to write, and how to measure whether it's working.
Here's what happens by default on most platforms.
A visitor asks something your bot wasn't trained on. The bot either invents an answer that sounds plausible (hallucination) or says something like "I'm sorry, I don't have enough information about that." Full stop.
The visitor is now more frustrated than if the chatbot didn't exist. They came to your site looking for something, got either a wrong answer or a wall, and now they have no idea what to do next.
That's the fallback failure. And it's the most common setup problem I see.
The fallback isn't a nice-to-have. It's the difference between a chatbot that builds trust and one that quietly erodes it.
A visitor asks something your bot genuinely wasn't trained on. "What's your refund policy?" when you haven't added your refund policy as a source. "Do you integrate with Zapier?" when your integrations page isn't in the training set. The bot doesn't have the information.
The fix here is usually a source update, not a fallback change. Before writing elaborate fallback copy, check your conversation logs for the last 30 days. If the same three questions are repeatedly triggering fallbacks, add those pages to your training sources. Fallback is for the residual. Not for your whole FAQ.
Someone asks the bot to write a poem. Someone asks about your competitor's product. Someone asks a deeply personalized question that requires a human ("I need to upgrade but I have a weird billing situation"). These questions have no answer in your docs because no answer in your docs would serve them.
This is where fallback copy matters. The visitor needs to be handed off to something that can actually help them.
Three parts. All three. No exceptions.
1. Acknowledge the miss, briefly.
"I don't have that in my sources" is honest and fast. Don't over-apologize. Two words of acknowledgment, not three sentences. The visitor already knows the bot didn't answer; you don't need to dwell on it.
2. Route to a specific next step.
Not "contact our team." That's a dead end with no contact details. Specific means: an email address, a calendar link, a support inbox URL. The more specific, the better the handoff conversion. There's a real difference between "email us" (vague, friction) and "email me directly at saurav@5minbot.com and I usually reply same day" (specific, credible, human).
Name a real person if you can. "Contact support" converts worse than "email Saurav."
3. Set an expectation.
"I typically reply within a few hours during business hours." One sentence. No obligation, but it gives the visitor a reason not to abandon. Without it, they don't know if they're sending their question into a void or getting a response tomorrow.
A full fallback message, assembled:
I don't have that in my docs. Email me at saurav@5minbot.com and I'll get back to you within a few hours during IST business hours.
42 words. Takes 5 seconds to read. Does the job.
A properly built RAG chatbot doesn't just return answers. It returns answers with a retrieval confidence score — a number that says how closely the retrieved chunks matched the question.
When that score is below a threshold (typically around 0.3 on a cosine similarity scale), the bot doesn't have a strong retrieval match. This is exactly when hallucinations happen. The model got a bunch of loosely-related chunks, can't find the specific answer, and fills in from pretraining.
Most platforms let you configure what happens when retrieval confidence is low. Set this to "trigger fallback" instead of "attempt answer." You will lose a small percentage of conversations that might have been borderline-answerable. But you will stop the hallucinations that are quietly destroying your trust scores.
In saavos, the similarity threshold controls this directly. Anything below 0.3 triggers the fallback response instead of generating from weak context. That single setting is responsible for most of the difference between a 1–4% hallucination rate and a 15–20% one.
After going live, pull your conversation logs every day for the first week. You're looking for three things:
Fallback triggers on answerable questions. If your bot is falling back on "what does the Pro plan include?" — that's a sourcing problem, not a fallback problem. Add the pricing page to your sources and re-ingest.
Questions being answered from wrong sources. "Our return window is 30 days" might be right, but if the citation is pointing to a blog post instead of your returns policy, you have a retrieval quality problem. Tighten your source list.
Questions that genuinely can't be answered by docs. These are your true fallback cases. Review the handoff copy and make sure the routing is working. Are people actually emailing you after hitting fallback? You can tell: check your inbox.
First-week log review usually surfaces 3–5 source gaps. Fix those and fallback volume drops 40–60% because the bot now answers what it previously couldn't. The genuine fallbacks that remain are the ones your copy and routing actually have to handle.
"I'm sorry I couldn't help with that." No routing. No next step. Visitor is stuck.
"Please visit our FAQ page." If the visitor is talking to your chatbot, they probably already couldn't find it. Sending them to a static FAQ they didn't read is not a handoff, it's a loop.
Dead contact links. Routing to a contact form that hasn't been tested in six months. Routing to an email address you check every other week. The handoff is only as good as what's on the other end of it.
Fallback on every question. If your fallback rate is above 30%, your training sources are wrong, not your fallback copy. The fallback message is for the irreducible residual. If it's triggering on common questions, fix the sources first.
A healthy fallback rate for a well-trained single-domain chatbot is 10–20%. That means 80–90% of questions get answered directly from sources.
Above 30%: your sources are too narrow or structured incorrectly. Add the pages your visitors actually need.
Below 5%: check for hallucinations. Very low fallback rates sometimes mean the bot is answering too confidently from weak retrieval instead of appropriately refusing.
Aim for 10–20% with a clear, specific fallback that routes to a real human. That's the target.
I run the support bot on saavos.com. Fallback copy is:
I don't have that in my docs. Email me at saurav@5minbot.com — I reply same day during IST business hours.
My fallback rate as of this week is around 14%. The questions that do fall back are mostly "do you integrate with X platform" (a few I haven't added yet) and billing edge cases. Both of those I genuinely want to handle personally anyway.
The bot handles roughly 85% of questions automatically. The other 15% come to my inbox as warm leads — they've already engaged with the bot, they have a specific question, and they're motivated enough to email. These are the best kinds of support conversations.
That's the actual goal of a good fallback. Not zero fallback. Zero fallback is how you get a hallucination problem. The goal is a fallback that converts miss-conversations into real ones.
Related: How to prevent AI chatbot hallucinations covers the five controls that reduce the hallucination rate before fallback is even needed. How to deflect 40% of SaaS support tickets has the source selection guide that reduces fallback volume by training on the right pages first.
If you're running a chatbot on your site and the fallback is still the default "I'm sorry I couldn't help" copy, that's the first thing to fix. Takes five minutes and measurably improves visitor trust.
Get the next post in your inbox
Honest writing on building, embedding, and shipping AI chatbots. No spam. Unsubscribe anytime.
A good fallback has three parts: acknowledge the miss briefly ('I don't have that in my docs'), route to a specific next step (email address, calendar link, or support URL — not just 'contact our team'), and set a response-time expectation ('I typically reply within a few hours'). Forty words beats a three-sentence apology. The visitor needs somewhere to go, not a sympathy message.
A well-trained single-domain chatbot should fall back on 10–20% of questions. Above 30% means the training sources are too narrow — add the pages visitors actually need. Below 5% is a warning sign: very low fallback rates sometimes mean the bot is answering from weak retrieval instead of appropriately refusing. The goal is not zero fallback. The goal is a fallback that routes the residual misses to a real human.
Pull conversation logs weekly and look for repeated fallback triggers. If the same three questions trigger fallbacks, those pages are missing from your training sources. Add them and re-ingest. Most chatbots in the first week have 3–5 source gaps that account for 40–60% of fallback volume — fixing those drops the fallback rate without touching the fallback message itself.
The similarity threshold is a minimum confidence score for retrieval. When vector search returns chunks that are too loosely related to the question, the score falls below this threshold. Setting the bot to trigger fallback below the threshold (around 0.3 on cosine similarity) prevents the model from hallucinating an answer from weak context. Without this setting, low-confidence retrievals become the source of most hallucinations.
Two causes: the system prompt is set to 'always be helpful' (which forces the model to generate something even when retrieval found nothing relevant), or the similarity threshold is too low (so the bot attempts to answer from weakly-matched chunks). Fix both: rewrite the system prompt to reward explicit refusal, and raise the similarity threshold so low-confidence queries trigger fallback instead of generation.
Builds tools for solopreneurs and small SaaS teams who don't have an afternoon to spare.
Paste your URL. Train your bot. Drop one script tag. No credit card.