Your AI Sounds Confident. That's Not the Same as Right.

Part 4 of a 5-part series on AI myths and misconceptions. Previously: Can Your Customers Still Reach a Human?

A grieving customer went to Air Canada's website to book a last-minute flight for his grandmother's funeral. He asked the airline's chatbot about bereavement fares. The bot told him, clearly and confidently, that he could book now and apply for the discount within 90 days.

It was wrong. Air Canada's actual policy did not allow that. When he applied, the airline refused, and pointed him to the correct policy sitting on another page of its own website.

He took them to a tribunal, and he won. Air Canada argued it shouldn't be responsible for what its own chatbot said, that the bot was somehow a separate thing. The tribunal called that argument remarkable and ordered the airline to pay (Moffatt v. Air Canada, 2024).

Here's the part that matters for you. The bot didn't sound unsure. It didn't hedge. It handed over a wrong answer with the exact same confidence it uses for a right one. And the business was on the hook for it.

Why It Actually Happened

Take someone off the street, zero training, and put them behind a customer service desk. What are the odds they give a customer the exact right bereavement fare policy? Close to zero. Not because that person is incapable of the job. Because nobody ever taught them the job.

That's exactly what happened here. The chatbot wasn't broken. It was untrained, and it still got put in front of a customer and trusted with an answer it was never actually taught. AI predicts the most likely-sounding response based on patterns in what it's seen. When it hasn't been properly trained on a business's real, specific policies, its best guess can come out sounding exactly as confident as a correct answer, even when it's flatly wrong. People in the field call this “hallucination.” It's really just an untrained employee, dressed up in a calm, steady voice.

A system that's actually been taught the real policy doesn't have this problem within what it knows, the same way a properly trained employee doesn't guess at the bereavement fare rule, they just know it, because someone did the work of teaching them. The failure here was never the technology. It was skipping the training and trusting the answer anyway.

This is not a reason to avoid AI

It's a reason to be clear about which jobs it can hold on its own and which ones need a person's eyes before anything reaches a customer.

Build the knowledge base to the real, exact scope of the business, every service, every price, every policy, and AI can answer confidently within that scope because it was actually trained to know it, not because a wrong answer there would supposedly be cheap.

Two things still route to a person, and neither is a judgment call about how bad a mistake would be. If someone asks for a human, that gets honored immediately, no arguing, no extra loop first. And if a question falls genuinely outside what the system was actually built to know, something the business itself doesn't have a standard answer for, the AI hands it off instead of guessing at something it was never taught.

Frequently Asked Questions

Can't I just tell the AI to say when it's unsure?

Telling it to hedge more doesn't fix the real problem. What actually matters is whether the system has a genuine, defined scope it was properly trained on. Inside that scope, it shouldn't need to hedge at all, it should just know the answer. Outside that scope, a more careful-sounding guess still isn't the fix. A clean handoff to a person is.

How do I know which answers actually need a human check versus which don't?

It's not about how costly a wrong answer would be, it's about scope. If the knowledge base was actually built to cover it, the system should answer directly and accurately, no hedging needed. A human check comes in for two reasons only: the customer specifically asked for one, or the question genuinely falls outside what the system was trained to know.

Is this only a real risk for a big company like Air Canada, or does it apply to a small business too?

It applies more directly to a small business, not less. Air Canada could absorb a $650 judgment without blinking. A local business doesn't have that same cushion, and a lost customer from one bad answer often costs more than the case itself did. A Chicago business attorney put it plainly in a 2026 analysis of the ruling: it applies far beyond the airline industry, any business that puts an AI tool in front of customers owns what that tool says, the same way it would own a promise made by an employee on the phone.

A simple next step

Make a short list of what your knowledge base actually covers right now, and be honest about any real gaps in it.

Anywhere there's a gap, or anywhere a customer might reasonably want a person instead, that's where the handoff needs to be clean and immediate, not a guess dressed up as an answer.

If it would help, I'll go through that with you and make sure the system's actual scope matches what it's supposed to know, with a clean handoff built in for anything outside it. No obligation, and the map is useful no matter who ends up building your systems.

Continue to the final piece in this series: The One Question to Ask Before You Let AI Do Anything.

Sources & Further Reading

Moffatt v. Air Canada, 2024 BCCRT 149, British Columbia Civil Resolution Tribunal

Can Someone Sue Your Business Over an AI Hallucination?, George Bellas, Business Attorney Chicago