Executive Summary
Legal teams at law firms advising financial market infrastructure clients rely on the CPMI-IOSCO 2016 Cyber Resilience Guidance as a primary reference point for understanding regulatory expectations around cyber risk governance, incident response, and perimeter security. Our research tested AI tools against targeted questions drawn from this guidance and found that AI assistants produced incorrect answers on questions where the correct response required distinguishing between closely related CPMI publications from the same period.
The single finding recorded here involves an AI tool confidently attributing a specific strategic phrase to the wrong CPMI document — and then, when pressed, acknowledging uncertainty about the attribution rather than correcting it with a verified source. For a legal team, this type of error is particularly hazardous: it looks authoritative, survives casual review, and will only surface as a problem when a regulator, counterparty, or opposing counsel checks the underlying document.
How AI gets this regulation wrong
The failure pattern recorded against this regulation is one of confident misattribution: AI tools presented incorrect source information as settled fact, and only disclosed uncertainty when directly challenged. The table below maps this failure mode against the specific finding, showing how an AI answer that appeared precise and well-sourced turned out to cite the wrong document entirely.
| AI's Failure Mode | Count | Affected findings |
|---|---|---|
| Exposed Fabrication | 1 | Finding#1 |
What that means for your team
For legal teams at law firms, the dominant risk arising from this finding is the delivery of work product — advice, memos, or client-facing materials — that rests on a misidentified regulatory source. The table below translates the AI failure into practical exposure categories: where it touches drafting, regulatory mapping, and the professional-responsibility obligations that attach when a law firm's advice is factually incorrect.
| Risk Impact | Count | Affected findings |
|---|---|---|
| Wrong deliverable | 1 | Finding#1 |
When this affects your department
Legal teams at law firms encounter the CPMI-IOSCO 2016 Cyber Resilience Guidance most frequently when advising financial market infrastructure operators — central counterparties, central securities depositories, payment systems — on their regulatory obligations. This arises in contexts such as drafting cyber resilience frameworks, reviewing incident response policies against international benchmarks, advising on FMI licensing or authorisation applications where cyber standards form part of the supervisory assessment, and conducting due diligence on FMI-adjacent counterparties or acquisition targets. The guidance is also regularly cited in cross-border matters where a law firm is mapping the regulatory requirements of multiple jurisdictions against CPMI-IOSCO principles.
The specific risk this finding introduces is the conflation of distinct CPMI publications from the same era. If an AI tool incorrectly attributes a strategic phrase or concept to the 2016 Cyber Guidance when it actually originates in a different CPMI document — a speech, a follow-up strategy paper, or a related payment fraud report — the legal advice built on that answer will cite the wrong source.
In a regulatory submission, a legal opinion, or a client memo, a sourcing error of this kind is not a minor footnote issue: it undermines the credibility of the entire analysis and may cause the advice to misstate what the 2016 Guidance actually requires.
For international law firms operating across multiple jurisdictions, the compounding risk is that their clients may use the AI-assisted legal advice to populate internal compliance documentation that is then reviewed by national regulators who are closely familiar with the CPMI text. A regulator that identifies a fabricated or misattributed citation in a client's compliance submission is likely to question the rigour of the firm's entire regulatory mapping process — exposing the firm to professional reputational damage beyond the immediate correction.
The findings at a glance
The table below summarises each finding recorded against this regulation for legal teams at law firms, showing the question topic, the AI failure type, and the risk category it creates for the firm.
| # | Finding title | Type | Citation ID |
|---|---|---|---|
| 1 | Misattribution of CPMI cyber strategy phrase to wrong document | Hallucination | RLB-F-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q014 |
Aggregate impact
The finding recorded against this regulation points to a specific and recurring vulnerability in AI-assisted legal research on CPMI materials: the CPMI published a cluster of related cyber and payment security documents around 2016–2018, and AI tools struggle to distinguish between them reliably. The 2016 Cyber Resilience Guidance, the 2018 wholesale payments fraud and endpoint security strategy, and speeches by senior CPMI officials from the same period share overlapping themes and are frequently cited together in secondary literature.
This proximity means that an AI tool can produce an answer that sounds correct — referencing genuine CPMI work — while misattributing the specific source.
For legal teams, the systemic risk is not just in the single misattribution identified here. It is in the broader pattern that suggests AI tools are unreliable when asked to distinguish the precise textual content or provenance of closely related CPMI documents. A legal team that uses AI to shortcut the process of mapping regulatory language back to its source document cannot assume the AI has correctly identified which document a phrase, concept, or requirement actually appears in.
Law firms advising FMI operators across international jurisdictions face particular exposure because the CPMI-IOSCO framework is a global reference standard — regulators in multiple jurisdictions cite it directly when setting national cyber resilience requirements for FMIs. Any error in characterising what the 2016 Guidance does or does not say can propagate across multiple client matters, multiple jurisdictions, and multiple regulatory submissions before it is identified.
What your team should do
The default position for legal teams using AI tools on CPMI-IOSCO materials should be: treat any AI-supplied citation — a phrase, a section reference, a document title, a date — as unverified until checked directly against the BIS website. The CPMI-IOSCO 2016 Cyber Resilience Guidance is publicly available at bis.org, and the full text is short enough that a targeted keyword search takes minutes. If the AI has attributed a phrase or principle to this document, check that it actually appears there before including it in any work product.
For questions that require distinguishing between CPMI publications — what a specific document says versus what a related speech, strategy paper, or follow-up report says — AI tools should not be used as the primary source. The finding here shows that AI tools can produce plausible-sounding but incorrect attributions even when challenged, which means the usual safeguard of querying the AI further does not reliably surface the error. Legal researchers should go directly to the BIS publications index when the question is specifically about which document a given principle or phrase originates from.
AI tools remain useful for legal teams working with this regulation in lower-stakes contexts: structuring a preliminary overview of CPMI-IOSCO cyber principles before reading the document, identifying the general topics covered, or drafting questions to pose to a compliance specialist. They are also reasonable starting points for understanding the regulatory landscape before the detailed legal research begins. The boundary is citation-level accuracy: any AI-generated claim about what a specific CPMI document says, requires, or uses as a term of art must be verified against the primary source before it enters advice, a memo, or a regulatory submission.
How RLB Can Help
RegLeg's published Hallucination Research gives Legal teams at law firms a ready pre-flight check before placing weight on AI-assisted output in regulatory matters. Each research entry documents a confirmed failure mode against a specific instrument — the type of provision involved, how the AI went wrong, and the risk consequence — so lawyers can run a quick cross-reference against the regulation they are working with before finalising advice, drafting submissions, or briefing clients. The research is freely available and requires no engagement to access.
For firms that want to go further, RLB offers bespoke regulator deep-dives scoped to the specific bodies and instruments your Legal function works with most. These engagements map which AI-supported workflows — regulatory research, precedent checking, cross-border compliance comparison, client advice drafting — carry the highest hallucination exposure in your practice context, and produce a ranked risk register the team can act on immediately. The output is confidential and is tailored to the jurisdictions and regulatory perimeters your firm operates across.
RLB also conducts confidential reviews of existing AI-use policies against its failure-mode catalogue, identifying gaps between the controls a firm has documented and the classes of error its AI tools are most likely to produce on regulatory questions. Each review closes with a prioritised remediation plan. Alongside policy work, RLB can supply training materials and CPD-aligned content — structured around real failure cases — that Legal teams can deploy internally to build consistent, defensible AI literacy across practice groups and seniority levels.