Set the authority boundary for customer-facing answers

Testing a customer assistant’s refusal of abusive requests covers one part of its behaviour. The acceptance review also needs to test prices, policies and entitlements. A polite answer can still give the customer a materially false instruction.

Assess the consequences of the assistant’s statements, including whether a person may rely on them to purchase, claim or change something. Content filters, factual support and approval authority address different properties. They need distinct acceptance checks.

Treat customer-facing statements as part of the organisation’s service responsibility. Their legal effect depends on jurisdiction, wording and circumstances. The design should identify which statements require authoritative data or approval rather than assume a disclaimer removes every consequence.

Two incidents that are usually told as one

They are told together because both involve a chatbot behaving badly, and they are worth separating because only one of them created a liability.

1. December 2023 A dealership assistant agrees to a one-dollar sale No reported enforcement A visitor instructed the assistant to agree with everything and to call its answers binding. It did. The dealership declined to honour it and nothing was enforced.
2. February 2024 Moffatt v Air Canada Liability found A tribunal held the airline responsible for a bereavement-fare policy its assistant described and the airline did not have.
3. 2 August 2026 EU AI Act transparency obligations apply In force A person interacting with an AI system has to be told, unless it is obvious from the circumstances.

The first is the famous one. In December 2023 a visitor to a Chevrolet dealership site told the assistant to agree with anything the customer said and to end every answer with a line about the offer being binding, then asked to buy a Tahoe for a dollar. The assistant complied. The exchange is recorded in the AI Incident Database alongside the other things visitors got that assistant to do, including recommending a competitor. The cited report does not describe a completed sale or enforced one-dollar agreement.

The second is the one that matters. In Moffatt v Air Canada, the British Columbia Civil Resolution Tribunal found the airline liable for negligent misrepresentation after its website assistant described a bereavement-fare refund process the airline did not operate. The airline argued, in substance, that the chatbot was responsible for its own statements. The tribunal did not accept it, holding the airline accountable for the information on its own website whichever component produced it, and awarded the passenger damages — the American Bar Association summary is the clearest short account of the reasoning.

The ruling concerns negligent misrepresentation on the facts of a Canadian dispute. It is useful context for reviewing a website assistant, but it does not settle liability for every generated statement or require every organisation to honour any offer a chatbot produces.

Validate consequential statements

Filtering and prompt instructions can reduce some failures. They do not alone establish that a price, policy or entitlement is correct. User input and retrieved text can influence generation despite their intended lower authority. Validate consequential content through trusted records and controls appropriate to the task.

Specify what is checked before a statement reaches the customer, what happens when a source is missing or conflicting, and who may approve an exception. Review the effect of an incorrect statement as well as its likelihood.

If this system states a price, a policy or an entitlement that is wrong, what happens next?

• Nothing — the statement stands on its own The assistant is a commitment channel. Treat every output as published copy. This is the position an organisation ends up in by default, without deciding to.
• It is checked against a system of record before it reaches the customer The assistant explains. The record commits. Entitlements, prices and policies are looked up, never generated.
• A person confirms anything that creates an obligation The assistant drafts. A named human commits. Only real if the confirmation step has the information and the time to be a decision.

A record lookup supplies authoritative values only if the record is current, permitted and correctly used. Human confirmation supplies meaningful review only if the person has the necessary information and authority. Test both paths, including missing sources and disagreement, rather than treating either as sufficient evidence by itself.

What to put in front of the launch decision

The disclosure line is now a legal obligation rather than a courtesy for systems in scope of the EU AI Act. Article 50 requires that people are informed they are interacting with an AI system unless that is obvious from the circumstances, and those transparency duties have applied since 2 August 2026 — they were not moved by the deferrals that pushed several high-risk obligations later. The text of Article 50 is short and worth reading in full before somebody paraphrases it into a project plan.

Agree which records are needed for a disputed statement and how long they may be retained. A useful reconstruction connects the conversation, evidence, versions and subsequent action. Protect access and record gaps. Keeping an application log does not alone establish that the required case can be reconstructed.

What this does not tell you

This is not legal advice, and Moffatt is a decision of a Canadian tribunal about a Canadian contract. It does not transplant into English law, or into anyone else’s, as a rule. What it does is establish the argument an organisation will be running if it ever needs to disclaim its own assistant, and that argument has now been tested once and lost. Formal interpretation for your jurisdiction and your terms stays with your counsel.

A customer assistant may be useful within a defined statement and action boundary. Evaluate the intended benefit alongside incorrect information, unauthorised commitments and handoff capacity. The review should establish that boundary for the actual service, rather than assume every assistant is either harmless or unsuitable.

The customer-service owner should agree which statements the assistant may make, their evidence source and the approval path for consequential exceptions. Review a sample of correct, unsupported and disputed responses before accepting the release scope. Keep accountability connected to the customer outcome.