This case study examines how AI tools respond to regulatory questions relevant to Compliance teams at Retail Banking firms operating under international frameworks. It covers one regulation tested: the Guidance on Cyber Resilience for Financial Market Infrastructures issued by CPMI-IOSCO in 2016. Across the aggregated findings, one question was identified where multiple AI tools produced materially incorrect responses. The errors documented here are not edge cases — they arise from queries that Compliance professionals routinely pose when verifying the current status of operative international standards.
These findings are intended to help Compliance teams calibrate where AI tools can and cannot be trusted in their day-to-day regulatory work.
Compliance teams at Retail Banking firms regularly consult AI tools when verifying the currency of international regulatory standards — particularly when scoping obligations for new product launches, updating third-party risk frameworks, or briefing senior management and board-level committees on the regulatory landscape affecting financial market infrastructure. The CPMI-IOSCO Cyber Resilience Guidance sits squarely in this territory: it is referenced in supplier due-diligence questionnaires, internal cyber governance policies, and regulatory mapping exercises conducted when a Retail Banking firm assesses its exposure to FMI-linked cyber risks.
An AI tool that confidently misrepresents whether that guidance is still operative — or whether it is under active revision — will produce errors that flow directly into those downstream work-products.
The corporate use-cases are concrete. A Compliance team drafting an internal policy on third-party FMI dependencies may ask an AI tool whether the 2016 guidance remains the primary standard. A regulatory affairs analyst preparing a board paper on international cyber resilience expectations may use an AI-generated summary as a basis for their briefing. A firm responding to a regulatory inquiry about its FMI-related cyber controls may rely on AI-assisted research to confirm which standards are in force.
In each of these scenarios, the AI's answer shapes a work-product that may be presented internally as settled fact, passed to a regulator, or used to justify a compliance position.
If the AI's answer is wrong — and in the findings below it is — the firm carries the cost. Regulatory action, including formal inquiry, requires remediation and may result in public findings where a firm has misrepresented the regulatory position it was operating under. Operationally, policies and controls built on a superseded or incorrectly characterised standard will require rework. Reputationally, a Retail Banking firm that presents outdated regulatory mapping to a regulator or counterparty faces credibility damage that is disproportionate to the original error.
The individual employee who posed the AI query is rarely the focus of that accountability — the department, its leadership, and ultimately the firm absorb the consequences.
The single finding aggregated here reveals a specific and consequential failure mode: AI tools asserting with confidence that an international regulatory standard is still fully operative when it is, in fact, under active revision. Both AI tools tested gave substantively identical incorrect responses — each stating that the 2016 CPMI-IOSCO Cyber Resilience Guidance had not been formally revised or superseded, when a consultative document for updated guidance had been published just weeks earlier. The error is not a subtle mischaracterisation; it is a categorical misstatement of regulatory status.
The AI tools presented a frozen view of the regulatory landscape as though it were current fact.
The pattern clusters tightly on a single regulation and a single regulator — CPMI-IOSCO and the BIS — but the underlying dynamic is broadly applicable. AI tools trained on data predating a recent regulatory development will not spontaneously flag that their knowledge may be stale. They will instead answer with the same apparent authority they would bring to a settled question. For a Compliance team that has not independently verified the currency of the AI's sources, this creates a credibility trap: the more confident and well-structured the AI's response, the less likely the team is to question it.
The systemic risk for a Retail Banking firm is a function of how many downstream work-products rest on a single AI-generated answer. If the incorrect assertion that the 2016 guidance is still operative enters one board paper, one supplier questionnaire, or one regulatory submission, the error replicates silently across everything built on that foundation. Remediation — once the error is identified — requires not only correcting the immediate work-product but auditing every other output that may have incorporated the same AI-sourced position.
The compounding cost of that audit, and the reputational exposure of having presented incorrect regulatory mapping to an external audience, is substantially larger than the original query would suggest.
1 finding in this case study. Click any to see its full evidence card.
The default position for any Compliance team at a Retail Banking firm should be that AI tools are a starting point for regulatory research, not a primary source. This is especially true for questions about the current status of international standards, where the regulatory position can shift — through consultation papers, revised guidance, or superseding publications — in ways that AI tools may not reflect. Before any AI-generated answer about regulatory currency is relied upon in a work-product, it should be verified against the relevant regulator's own portal or official communications channel.
For CPMI-IOSCO matters, that means checking the BIS website directly. No AI tool, however well-structured its response, substitutes for that step.
At the firm level, several practical safeguards reduce the risk of AI errors propagating into consequential outputs. A regulatory-verification policy that explicitly identifies AI tools as unreliable sources for questions of regulatory status — as distinct from regulatory background or explanatory content — gives Compliance staff a clear operating principle. Audit trails for any AI output that influences a firm work-product, combined with sign-off requirements before AI-generated regulatory positions enter firm-wide use, create accountability checkpoints that surface errors before they replicate.
Where AI-assisted content enters regulatory-facing material, it should be clearly distinguished — internally at minimum — from content that has been independently verified against primary sources. These are not burdensome controls; they are proportionate responses to a documented failure pattern.
AI tools do have a productive role in Compliance workflows. Drafting non-regulatory copy, generating structured first-draft questions for further research, summarising long documents that the team will then verify, and producing initial frameworks for regulatory mapping exercises are all areas where AI assistance adds value without the same risk profile. The discipline is in knowing where the boundary sits: AI tools can accelerate the research process, but the determination of what a regulation currently requires — and whether it is still in force as stated — remains a human verification task.
RegLeg's published hallucination research gives Compliance teams at Retail Banking firms a free, ready-to-use reference before relying on any AI answer in the regulatory areas covered by our findings. When a team member is about to use an AI-generated response about international cyber resilience standards — or any other regulatory topic in our catalogue — they can check whether that topic has already been tested and documented. Where it has, the research sets out precisely what AI tools get wrong, what the regulator actually says, and where to verify.
This is a practical pre-use check, not a theoretical caution: it takes minutes and directly addresses the failure modes that surface in real Compliance workflows.
For firms that want a deeper understanding of their specific exposure, RegLeg offers bespoke regulator deep-dives tailored to the workflows of a Retail Banking Compliance function. These map which AI-supported tasks — regulatory mapping, policy drafting, supplier due-diligence, board reporting — carry the highest hallucination risk for the firm's particular regulatory footprint, and identify the rule areas where AI tools have the worst track record. The output is a prioritised risk picture the Compliance team can act on directly, rather than a generic caution about AI limitations.
RegLeg also offers confidential review of a firm's existing AI-use policy against our failure-mode catalogue, with prioritised recommendations for where controls need strengthening or where the policy is silent on documented risks. For Compliance teams building or refreshing internal AI governance frameworks, we can provide training material and CPD-aligned content that staff can use directly — grounded in real regulatory findings rather than abstract AI literacy guidance. Our aim is to work alongside the Compliance function as a research partner, not to add friction to the use of AI tools that genuinely add value.