Put a decision gate on live AI testing

Consider a financial-services system that routes customer messages accurately in a test set. Before it reaches live traffic, the organisation still needs to decide which customers and message classes are in scope, what happens when routing fails and who can suspend the service. A model score alone does not answer those questions.

Set a decision gate between the proof of concept and live use. Agree the intended outcome, cohort, evidence, guardrails and stop authority before the trial begins. The resulting plan should allow the board to distinguish a case for expansion from a case for narrower testing, redesign or retirement.

What the FCA is actually testing

The FCA describes AI Live Testing as a programme for firms with mature proofs of concept ready for controlled market conditions. It distinguishes an AI system from an AI model and considers deployment context, governance, human involvement, evaluation, and input and output controls. Its programme moves through discovery, framework validation and AI system testing. Those are features of the FCA initiative, not a general-purpose approval pathway for every firm’s product.

The FCA’s AI sandbox serves an earlier kind of work: experimenting and developing AI propositions in a controlled environment. The two routes illustrate a maturity question a firm can ask independently of applying to either programme. Are the product and controls sufficiently defined to learn from real-world conditions, or is the organisation still discovering whether the use case is viable?

For the customer-message system, a sandbox may help establish that the model can process the data and support the desired task. Live testing asks whether the entire service behaves acceptably with actual queues, staff, customers and exceptions. That includes which messages it does not handle, how it passes a case to a human and what customers experience while an error is corrected.

Write a release decision before launch

The decision owner should specify the cohort and exposure boundary. Start with the kinds of messages, channels and customer groups for which the assessment is valid. Exclusions are as important as inclusions: a tool tested on routine questions has not thereby been tested on complaints, vulnerable customers or urgent loss reports.

Next, record the incumbent process. Measure response time, accuracy, repeat contacts, complaint escalation and manual effort before the AI path changes them. Choose one primary outcome and guardrails that represent customer harm, not only operational savings. Keep a comparison for long enough to observe the relevant outcomes, and preserve the ability to attribute a decision to the system version and human action that produced it.

Name stop triggers that a frontline team can execute. A concerning error pattern, missing trace, drift in the case mix or failure of the handoff may require a pause before the next governance meeting. Give a named role the technical ability to switch to the manual route. Rehearse that route once before the test begins.

Baseline

Response time, correct routing, repeat contacts and manual effort for the same message classes under the current process.

Outcome

Faster resolution of routine messages at comparable or better routing accuracy and customer experience.

Guardrails

Missed complaints or urgent cases, harm to vulnerable customers, unsupported answers, missing traces and failed human handoffs.

Decision rule

Start with routine messages only. Pause on a guardrail breach, investigate the trace and resume only after the owner accepts the remedy and a retest.

Choose an initial live slice that staff can inspect and support. In the message-routing example, routine questions could enter the AI path while complaints and urgent loss reports retain the existing route. Review automated and escalated cases, including missed handoffs. A material change in the case mix should trigger a fresh assessment rather than an automatic extension of the original result.

The pause route needs its own test. Who can disable the AI path outside office hours? Does the customer message reach a staffed queue immediately, or sit in a failed integration? Can the team identify which customers received an affected answer before the switch? A drill may reveal that the switch exists but the trace cannot be retrieved. That is a release blocker even if the model’s test score remains high.

At the gate, distinguish a controlled continuation from expansion. The board may have enough evidence to keep learning with the same cohort while lacking evidence for new customer groups or message types. Document that boundary, the next observation period and the person accountable for deciding again.

Present the review with both aggregate measures and relevant cases. Averages can obscure a group receiving poor service, while isolated examples cannot establish a general improvement. State the population and observation period, explain uncertainty and identify the evidence still needed for a wider release.

What this does not establish

Participation in an FCA programme does not certify a firm’s AI system or remove its regulatory responsibilities. Nor does this article say every firm must join one. The particular obligations depend on the service and the firm. Legal interpretation stays with counsel and the relevant compliance team.

The accountable executive needs a test plan that supports a decision and a credible means of stopping exposure. Agreement to another pilot is insufficient if the organisation cannot explain what it will learn or how that learning changes the release decision.