The FCA is launching AI Live Testing, and has published feedback on its Engagement Paper on the potential benefits, opportunities and challenges raised by its proposals.
On the whole, respondents welcomed the FCA’s proposal for AI Live Testing as a constructive and timely step. Key benefits and opportunities highlighted included:
- Real-world insights: live production testing was seen as a valuable mechanism for understanding how AI models perform under real-world conditions;
- Overcoming Proof of Concept (PoC) paralysis: many firms reported that AI PoCs often demonstrate technical merit but fail to progress due to concerns such as regulatory uncertainty and skills shortages;
- From principles to practice: respondents noted a lack of guidance on how to operationalise and measure key AI principles such as fairness, robustness, safety and security. AI Live Testing could help bridge this gap by providing a structured, repeatable process for assessing performance;
- Creating trust: respondents emphasised that traditional assurance methods are insufficient in the face of rapidly evolving AI capabilities and that trust in AI must be intentionally designed and transparently demonstrated;
- Addressing first-mover reluctance: some firms hesitate to use AI in sensitive areas without greater regulatory clarity;
- Regulatory comfort: being given ‘regulatory comfort’, potentially through individual guidance or other tools, can substantially de-risk innovative AI use and encourage firms to bring beneficial products and services to market more quickly;
- Collaboration: respondents noted that AI Live Testing is a welcome step forward between the regulator and the industry to jointly navigate the challenges; and
- Model metrics: AI Live Testing can foster collaboration and help develop a shared technical understanding on complex AI issues such as model validation, bias detection and mitigation and ensuring robustness.
The majority of submissions (52 out of 67) came from a range of non-regulated firms, including consultancies, AI specialists, RegTech firms, universities and civil society organisations, startups, trade associations and large tech providers. Of the 15 regulated firms that responded, there were high street and challenger banks, insurance firms, wealth managers, investment platforms, payment processing firms and credit reporting firms.
Applications for the first cohort of AI Live Testing close on 15 September 2025. The FCA will start working with those firms from October. Applications for the second cohort are due to open before the end of 2025, and the FCA aims to publish an evaluation report after 12 months. Currently, the FCA has plans for 2 cohorts made up of between 5 – 10 firms.