Questions to Ask Before You Buy an AI Chatbot

Ask any AI chatbot vendor what it does when it does not know an answer, who actually fixes it once it is live, and whether you can see what it says to your customers, before you buy anything. Every demo looks clean. The differences that matter do not show up in the sales call. They show up six months later, when something breaks and you find out the hard way what you actually paid for.

Why every vendor demo looks the same

Owners comparing a few AI chatbot vendors at once run into the same wall. Every pitch shows the bot handling one perfect question perfectly. Nobody shows what it does with a strange question, or what the bill actually looks like by month six. So people end up picking the vendor they liked best on the call, which is not a great way to buy something that is going to talk to your customers all day without you in the room.

Here is what actually separates a real build from a good sales pitch.

Does it write to your real calendar, or just say it does

Ask to see it book an actual open slot on your actual calendar, live, on the call. Not a screenshot. Not "it integrates with most major calendar tools." A demo where the bot just says it booked you in a chat bubble has not proven anything. If it cannot do the real thing while you are watching, it will not start doing it after you have paid.

What happens when it does not know the answer

This is the single best question to ask, and almost nobody asks it. Give the vendor a question a real customer would ask, one that sits outside whatever the bot was trained on. Watch what it does next. A build that is actually ready says it does not know and hands the conversation to a person. One that is not ready guesses.

A confident wrong answer in your own voice is worse than no answer at all, because it sounds like you said it.

We wrote more about how a bot should handle that exact moment in When Should an AI Chatbot Hand Off to a Human?.

Who keeps it working after you pay

Your prices change. Your hours change. You stop selling something you used to sell. Ask what happens to the bot when any of that happens on your end. Does it pull from something you already keep updated anyway, do you have to email the vendor every time, or is nobody watching it and you find out it is wrong from a customer complaint. A vendor with no real answer here is a vendor about to make you the one who maintains it, without ever saying that out loud.

This matters more the deeper the bot is wired into your other tools. We covered how an AI agent actually connects to your business tools and what tends to break first.

Can you actually see what it told your customers

Ask for a transcript log, not a satisfaction score. A vendor who is confident in what they built will show you exactly what the bot said to real people. A vendor who only hands you a summary number is asking you to trust them instead of showing you.

The short list to bring to any vendor call

• Ask it to book a real slot on your real calendar, live

• Ask it something it does not know, and watch what it does

• Ask what happens when your prices or hours change

• Ask to see an actual transcript, not a score

• Ask what happens to your setup if the vendor is gone in a year

If you want to see a real one before you go vendor shopping, we built a demo that runs on your own content. Paste a YouTube link and about a minute later you are talking to an AI version of yourself. It is free to try at demo.ajmarketingresults.com.

---

Common questions

Is an AI chatbot the same thing as an AI agent?

Not exactly. A chatbot usually answers from a small set of canned replies. An agent trained on your own content answers more like a person who read everything you ever wrote, and it can take an action, like actually booking the slot, instead of just replying about it. Ask which one you are actually being sold.

How much should this cost?

It depends more on the pricing model than the product. Some vendors charge per conversation, some a flat monthly fee, some by usage. We broke down how these actually get priced and what each model rewards.

What if the vendor will only show me screenshots, not a live demo?

Ask why. A build that genuinely works can be shown live, on the spot, with a question you make up on the fly. Screenshots are easy to stage and easy to cherry pick.

Do I need my content organized before I talk to a vendor?

It helps to have your FAQs, past support answers, and policies gathered somewhere, but a real vendor should be able to work with what you have and tell you plainly what is missing. If they will just take whatever you hand them with no questions at all, that is worth noticing.

What happens if the vendor disappears or stops answering?

Ask this before you sign, not after. Who owns the account, where the content actually lives, and whether someone else could keep it running if the vendor vanished tomorrow. If nobody on the call can tell you plainly, that is worth noticing too.