By Saurav · saavos
[!TLDR] When your chatbot hits a question outside its training data, it has three options: hallucinate an answer, say "I don't know" and dead-end the visitor, or route them somewhere useful. Most out-of-the-box setups do the first two. A properly configured fallback does the third. This post covers exactly what to configure, what to write, and how to measure whether it's working.
The actual problem when a chatbot can't answer
Here's what happens by default on most platforms.
A visitor asks something your bot wasn't trained on. The bot either invents an answer that sounds plausible (hallucination) or says something like "I'm sorry, I don't have enough information about that." Full stop.
The visitor is now more frustrated than if the chatbot didn't exist. They came to your site looking for something, got either a wrong answer or a wall, and now they have no idea what to do next.
That's the fallback failure. And it's the most common setup problem I see.
The fallback isn't a nice-to-have. It's the difference between a chatbot that builds trust and one that quietly erodes it.
Two types of "can't answer" you need to handle differently
Type 1: Out-of-scope question
A visitor asks something your bot genuinely wasn't trained on. "What's your refund policy?" when you haven't added your refund policy as a source. "Do you integrate with Zapier?" when your integrations page isn't in the training set. The bot doesn't have the information.
The fix here is usually a source update, not a fallback change. Before writing elaborate fallback copy, check your conversation logs for the last 30 days. If the same three questions are repeatedly triggering fallbacks, add those pages to your training sources. Fallback is for the residual. Not for your whole FAQ.
Type 2: Genuinely out of bounds
Someone asks the bot to write a poem. Someone asks about your competitor's product. Someone asks a deeply personalized question that requires a human ("I need to upgrade but I have a weird billing situation"). These questions have no answer in your docs because no answer in your docs would serve them.
This is where fallback copy matters. The visitor needs to be handed off to something that can actually help them.
What a good fallback message contains
Three parts. All three. No exceptions.
1. Acknowledge the miss, briefly.
"I don't have that in my sources" is honest and fast. Don't over-apologize. Two words of acknowledgment, not three sentences. The visitor already knows the bot didn't answer; you don't need to dwell on it.
2. Route to a specific next step.
Not "contact our team." That's a dead end with no contact details. Specific means: an email address, a calendar link, a support inbox URL. The more specific, the better the handoff conversion. There's a real difference between "email us" (vague, friction) and "email me directly at saurav@5minbot.com and I usually reply same day" (specific, credible, human).
Name a real person if you can. "Contact support" converts worse than "email Saurav."
3. Set an expectation.
"I typically reply within a few hours during business hours." One sentence. No obligation, but it gives the visitor a reason not to abandon. Without it, they don't know if they're sending their question into a void or getting a response tomorrow.
A full fallback message, assembled:
I don't have that in my docs. Email me at saurav@5minbot.com and I'll get back to you within a few hours during IST business hours.
42 words. Takes 5 seconds to read. Does the job.
The thing most founders skip: the low-confidence threshold
A properly built RAG chatbot doesn't just return answers. It returns answers with a retrieval confidence score — a number that says how closely the retrieved chunks matched the question.
When that score is below a threshold (typically around 0.3 on a cosine similarity scale), the bot doesn't have a strong retrieval match. This is exactly when hallucinations happen. The model got a bunch of loosely-related chunks, can't find the specific answer, and fills in from pretraining.
Most platforms let you configure what happens when retrieval confidence is low. Set this to "trigger fallback" instead of "attempt answer." You will lose a small percentage of conversations that might have been borderline-answerable. But you will stop the hallucinations that are quietly destroying your trust scores.
In saavos, the similarity threshold controls this directly. Anything below 0.3 triggers the fallback response instead of generating from weak context. That single setting is responsible for most of the difference between a 1–4% hallucination rate and a 15–20% one.
The week-one log review is where you find the gaps
After going live, pull your conversation logs every day for the first week. You're looking for three things:
Fallback triggers on answerable questions. If your bot is falling back on "what does the Builder plan include?" — that's a sourcing problem, not a fallback problem. Add the pricing page to your sources and re-ingest.
Questions being answered from wrong sources. "Our return window is 30 days" might be right, but if the citation is pointing to a blog post instead of your returns policy, you have a retrieval quality problem. Tighten your source list.
Questions that genuinely can't be answered by docs. These are your true fallback cases. Review the handoff copy and make sure the routing is working. Are people actually emailing you after hitting fallback? You can tell: check your inbox.
First-week log review usually surfaces 3–5 source gaps. Fix those and fallback volume drops 40–60% because the bot now answers what it previously couldn't. The genuine fallbacks that remain are the ones your copy and routing actually have to handle.
Common fallback mistakes
"I'm sorry I couldn't help with that." No routing. No next step. Visitor is stuck.
"Please visit our FAQ page." If the visitor is talking to your chatbot, they probably already couldn't find it. Sending them to a static FAQ they didn't read is not a handoff, it's a loop.
Dead contact links. Routing to a contact form that hasn't been tested in six months. Routing to an email address you check every other week. The handoff is only as good as what's on the other end of it.
Fallback on every question. If your fallback rate is above 30%, your training sources are wrong, not your fallback copy. The fallback message is for the irreducible residual. If it's triggering on common questions, fix the sources first.
Fallback rate as a health metric
A healthy fallback rate for a well-trained single-domain chatbot is 10–20%. That means 80–90% of questions get answered directly from sources.
Above 30%: your sources are too narrow or structured incorrectly. Add the pages your visitors actually need.
Below 5%: check for hallucinations. Very low fallback rates sometimes mean the bot is answering too confidently from weak retrieval instead of appropriately refusing.
Aim for 10–20% with a clear, specific fallback that routes to a real human. That's the target.
What this looks like in practice
I run the support bot on saavos.com. Fallback copy is:
I don't have that in my docs. Email me at saurav@5minbot.com — I reply same day during IST business hours.
My fallback rate as of this week is around 14%. The questions that do fall back are mostly "do you integrate with X platform" (a few I haven't added yet) and billing edge cases. Both of those I genuinely want to handle personally anyway.
The bot handles roughly 85% of questions automatically. The other 15% come to my inbox as warm leads — they've already engaged with the bot, they have a specific question, and they're motivated enough to email. These are the best kinds of support conversations.
That's the actual goal of a good fallback. Not zero fallback. Zero fallback is how you get a hallucination problem. The goal is a fallback that converts miss-conversations into real ones.
Related: How to prevent AI chatbot hallucinations covers the five controls that reduce the hallucination rate before fallback is even needed. How to deflect 40% of SaaS support tickets has the source selection guide that reduces fallback volume by training on the right pages first.
If you're running a chatbot on your site and the fallback is still the default "I'm sorry I couldn't help" copy, that's the first thing to fix. Takes five minutes and measurably improves visitor trust.