What Your AI Chatbot's Knowledge Base Is Missing
Most AI chatbot knowledge bases get built the same way. Pull the FAQ page, dump in a batch of old support tickets, call it done. That covers the questions people already bothered to write down somewhere. It misses the ones they only ask out loud, when nobody made them fill out a form first.
What Usually Goes Into a Chatbot's Knowledge Base?
The standard advice, and it's not wrong, is to pull from a specific list: your FAQ page, product manuals, internal wikis, policy documents, old support tickets, and chat logs. Quickchat AI's guide to building a chatbot knowledge base lays out that exact list and tells you to start by pulling your most frequently asked questions out of your support history.
That's a fine starting point. It's also where almost everyone stops. (If your FAQ page itself isn't cutting support volume, that's a related but separate problem, worth a read on its own: the issue there is usually the interface, not the source material.)
Why Support Tickets and FAQ Pages Miss the Real Question
A support ticket is something a person sat down and typed into a box. They picked their words, maybe trimmed the question down to sound less dumb, maybe only asked the part they thought support could actually help with. Zendesk's own advice for building a knowledge base is to audit your top 15 most-asked customer questions from that exact record. Useful, but it's still the filtered version of what people actually wanted to know.
A live question is different. It's the thing someone blurts out mid conversation once they've decided you're safe to ask. It's messier and usually closer to their real problem than anything that made it into a ticket.
Where the Real Questions Actually Show Up
Three places, and none of them are your help desk.
The unscheduled minutes after a workshop or a live call. The session ends, people stick around, and that's when the real questions come out. Not the ones on the agenda, the ones they were actually stuck on the whole time.
DMs, once someone stops treating you like a company. The first message is usually careful. A few exchanges in, the questions get blunt and specific, and that's the good material.
The second question on a support call, not the first. People open with the "official" reason they called. The follow-up, the one that slips out after they've been heard once, is usually the real one.
How to Actually Capture It
None of this requires new tooling. It requires paying attention to something you're already generating and not currently keeping.
Spend five minutes after every live session writing down what got asked in the informal window, even the small stuff. Read back through actual DM threads for the blunt follow-up questions instead of only the ones that led to a sale. On support calls, stop treating the second question as a tangent and start treating it as the thing worth logging.
None of that is glamorous. It's also the part almost nobody does.
How Much Do You Actually Need to Start?
Less than people assume. Quickchat AI's guide recommends starting with as few as five to ten well-formatted documents to prove the idea works, before you try to cover everything.
Start with a small set of your most critical, well-formatted knowledge documents, like five to ten key FAQ pages or a concise policy document, and validate the concept before building out full coverage.
You don't need a finished library. You need real material, even a small amount of it, over a polished FAQ page nobody actually wrote from real questions.
We Built This the Hard Way First
We built the member support agent for Brock Johnson's InstaClubHub community, and the first lesson was that members don't search. The answer could already be sitting in a course module, and they'd still ask in chat, because asking is faster than digging. A better FAQ page didn't fix that. An agent trained on the actual questions members asked, not just the course content, did.
FAQ
What's the real difference between a support ticket and a live question, for training purposes?
A ticket is the edited version, someone chose their words before typing it in. A live question is the raw version, asked mid conversation once they trust you enough to just say it. Both are useful. The live version is usually closer to the actual problem.
How many FAQ entries does an AI agent actually need before it's useful?
You can start meaningfully small, as few as five to ten solid documents or a focused set of real questions and answers, rather than waiting until you have a complete library.
Should old support tickets go into the knowledge base at all?
Yes, they're real answers to real questions and genuine training material. Just don't treat them as the whole picture. They're filtered by whatever the customer thought was worth typing into a form.
I've never run a workshop or a live call. Where do I find these questions?
DM threads and comment replies work the same way. Look for the follow-up question after the first one, that's usually where the real material is.
Does this replace a written FAQ page?
No. A written FAQ still has a job, it's just not the only, or even the best, source. Treat it as one input among several, not the finished product.
If you want to see what an agent trained on real content actually sounds like, there's a free demo where you paste in your own material, a YouTube link, course content, whatever you've got, and talk to a version of it trained on what you actually said instead of a generic script.
Comments