Executive Summary
The CPMI-IOSCO Guidance on Cyber Resilience for Financial Market Infrastructures (2016) is the primary international standard against which public auditors assess whether systemically important payment systems, central securities depositories, and central counterparties have adequate cyber risk governance and operational controls. Across five questions put to AI tools on this regulation, every response contained a hallucination — none were correct. The failures share a common shape: AI assistants converted genuinely uncertain or unconfirmed relationships between this guidance and other frameworks (NIST CSF, FSB Cyber Lexicon) into confident affirmative claims, and overstated the operational detail that the 2016 document itself actually contains.
For public auditors in international jurisdictions, where regulatory opinion letters, audit opinions, and scoping memos frequently depend on precise knowledge of what a standard requires and how it interacts with later instruments, accepting these AI responses without independent verification would produce materially incorrect work product.
How AI gets this regulation wrong
Every finding on this regulation reflects the same underlying pattern: AI tools gave confident, affirmative answers to questions where the honest answer was either uncertain or nuanced, and when pressed they acknowledged they had overstated what could be verified. The specific failure recurs across two distinct topics — the relationship between the 2016 guidance and external reference frameworks such as NIST CSF and the FSB Cyber Lexicon, and the level of operational detail the 2016 guidance actually contains — with AI assistants in both cases converting structural similarity or temporal proximity into claimed explicit alignment or derivation.
| AI's Failure Mode | Count | Affected findings |
|---|---|---|
| Exposed Fabrication | 3 | Finding#1 · Finding#2 · Finding#3 |
What that means for your practice
For public auditors, every hallucination identified on this regulation carries the same consequence: a wrong deliverable. Whether the error concerns which frameworks the 2016 guidance formally acknowledges, how its definitions align with later FSB standardisation, or what operational detail it actually specifies, an auditor acting on the AI's confident but incorrect characterisation risks issuing an audit opinion or advice memo that misrepresents the regulatory baseline. The table below maps each finding to the specific type of practice risk it creates for auditors working in international jurisdictions.
| Risk Impact | Count | Affected findings |
|---|---|---|
| Wrong deliverable | 3 | Finding#1 · Finding#2 · Finding#3 |
When this affects Public Auditors
Public auditors working in international jurisdictions encounter the CPMI-IOSCO 2016 Cyber Resilience Guidance at several recurring points in an engagement cycle. When scoping a new audit of a financial market infrastructure — a central counterparty, central securities depository, or systemically important payment system — auditors routinely use this guidance to establish the regulatory cyber resilience baseline against which controls are assessed. They may also consult AI tools when drafting advice memos on whether an FMI's cyber framework meets international expectations, or when preparing training materials for teams unfamiliar with the BIS/CPMI regulatory architecture.
The risks are highest when auditors use AI to understand how the 2016 guidance relates to other frameworks. Questions such as whether the guidance formally references NIST CSF, what level of operational detail it provides for incident response, or whether its definitions align with the FSB Cyber Lexicon are exactly the kind of cross-framework scoping questions that appear early in engagements and shape the entire audit programme.
An incorrect answer at that stage — for example, believing the 2016 document provides detailed incident response procedures when a subsequent FSB publication fills that gap — leads auditors to apply the wrong standard against which client controls are evaluated.
The consequences are compounded in international contexts where auditors must navigate multiple overlapping frameworks. If an AI tool incorrectly asserts that the 2016 guidance explicitly cross-references NIST CSF, an auditor may design an audit programme that treats NIST CSF compliance as a proxy for CPMI-IOSCO compliance — a conclusion that the guidance itself does not support. Similarly, treating the FSB Cyber Lexicon definitions as retrospectively consistent with the 2016 text, when that consistency is in fact uncertain, can produce opinion letters that misstate the regulatory position for clients and supervisors relying on those letters.
The findings at a glance
The table below summarises all five findings on the CPMI-IOSCO 2016 Cyber Resilience Guidance, each representing a question where AI tools produced a hallucinated response that a public auditor in international practice might reasonably rely on.
Aggregate impact
The five findings cluster into two distinct but related problem areas. Three concern the relationship between the 2016 guidance and external reference frameworks: whether NIST CSF is formally cited, and whether the FSB Cyber Lexicon definitions are consistent with those in the 2016 document. In each case, AI tools converted genuine uncertainty — or outright absence of confirmation — into affirmative claims of explicit alignment or derivation.
The remaining two findings concern the guidance's own content: AI tools overstated the operational detail that the 2016 document provides on cyber incident response and recovery, failing to acknowledge that a subsequent FSB publication from 2020 fills a gap that the 2016 guidance deliberately leaves open.
What unites all five findings is that AI assistants initially presented their incorrect answers with high confidence, and only acknowledged uncertainty when challenged. This pattern is particularly dangerous for public auditors because the questions involved — framework cross-references, definitional consistency, document scope — are precisely the preliminary scoping questions that auditors ask early in an engagement, often without yet having read the primary source. An auditor who receives a confident AI answer and moves on without verification may not revisit the point until a client or regulator identifies the error.
The systemic risk for international auditors is that the 2016 guidance sits at the centre of a web of related instruments — PFMIs, FSB Cyber Lexicon, NIST CSF, later CPMI and FSB supplements — and AI tools appear to treat that web as more tightly integrated than it actually is. The guidance was developed before several of those instruments existed, and the relationships between them are at best unconfirmed and at worst incompatible. Auditors who rely on AI characterisations of how these instruments interact risk designing audit programmes around assumed alignments that the regulators themselves have not established.
What your team should do
The default position for any public auditor using AI tools on the CPMI-IOSCO 2016 Cyber Resilience Guidance should be: verify at source before committing anything to a deliverable. The primary text is published by BIS at bis.org and is short enough to read in full. Questions about what the 2016 document explicitly says, which external frameworks it cites, and what level of operational detail it provides are all answerable by reading the document itself — these are not questions where AI tools add value, because the answers are not interpretive, they are factual and verifiable.
Given that every AI response tested on this regulation contained a hallucination, treating any AI characterisation of this document as a starting hypothesis rather than an established fact is the minimum safe practice.
For practical workflow, teams should establish a clear distinction between tasks where AI is useful and tasks where it is not. AI tools can assist with structuring audit programmes, drafting risk matrices, or generating first-draft questionnaires — tasks where the output requires human expert review regardless and where a factual error about this specific regulation is less likely to propagate undetected. They are not safe for answering precise questions about the guidance's content, its formal cross-references to other frameworks, or its definitional relationship with instruments published later.
Those questions should be answered by reading the source documents and, where relationships between documents are uncertain, recording that uncertainty explicitly in the audit file rather than resolving it with an AI assertion.
When advising FMI clients in international jurisdictions on how their cyber resilience framework maps to the 2016 guidance alongside NIST CSF, FSB Cyber Lexicon, or other instruments, auditors should avoid treating AI-generated cross-framework mappings as reliable. The BIS/CPMI regulatory architecture evolves through successive publications, and the 2016 guidance is a specific point-in-time document. Any mapping that claims explicit alignment between the 2016 text and frameworks published after it — or that asserts NIST CSF is formally adopted — needs a primary-source citation to support it. If the AI cannot provide one, the mapping should be treated as unconfirmed.
How RLB Can Help
RegLeg's published Hallucination Research gives public auditors a practical pre-flight check before placing weight on AI-assisted analysis of regulatory questions. The research catalogues the specific failure modes — misquoted thresholds, conflated jurisdictional requirements, fabricated citation trails — that AI tools produce most often in public-sector and cross-border audit contexts. Auditors can use these findings to calibrate their review steps before any AI output enters a working paper, providing a documented basis for the professional scepticism their standards already require.
Where an audit team or firm has multiple practitioners working across the same regulatory portfolio, RLB can deliver bespoke deep-dives on individual regulations. These sessions go beyond the published research to map failure modes specific to the instruments, guidance notes, and enforcement expectations most relevant to the team's current engagements. The output is practical rather than theoretical: teams leave with concrete review checkpoints aligned to the regulations they are actually auditing against.
RLB also develops training material and CPD-aligned content built around the failure-mode catalogue, so that auditors at all experience levels understand what to look for and why. For firms that have already deployed AI tools and drafted internal use policies, RLB offers confidential reviews of those policies against the same catalogue — identifying gaps between what the policy assumes AI tools will do reliably and what the research shows they frequently get wrong.