Executive Summary
Compliance teams at Management & Risk Consulting firms advising FMI clients on CPMI-IOSCO assessment obligations are increasingly turning to AI tools to accelerate the production of regulatory summaries, methodology notes, and engagement briefings — the kind of output that feeds directly into client submissions, regulatory response files, and board-level reporting. On the November 2025 Level 3 assessment of Principle 15 general business risk, AI assistants we tested produced factually incorrect timeline information: stating the assessment ran through 2024 when the primary document confirms engagement with FMIs continued through April 2025 and the assessment lifecycle extended to 2025.
The error is the type that travels invisibly into polished deliverables — a one-page methodology note drafted by AI, reviewed at speed, and submitted as factual context to a regulator or board carries the wrong dates without any visible flag. For a consulting firm whose credibility rests on being the authoritative source of record on assessments like this, a factual error in a client-facing regulatory submission is a reputational and professional-liability risk that a find-and-replace cannot fix after the fact.
How AI gets this regulation wrong
The failure we identified on this regulation falls into one of the most operationally dangerous categories: AI tools presenting outdated information as if it were current. On the CPMI-IOSCO Level 3 general business risk assessment, the AI adopted shorthand from secondary-source commentary — collapsing the full assessment lifecycle into a truncated date range — without flagging that the primary document tells a different story. The table below sets out how this failure mode manifests across the questions we put to AI assistants on this regulation.
| AI's Failure Mode | Count | Affected findings |
|---|---|---|
| Outdated | 1 | Finding#1 |
What that means for your team
When an AI-produced factual error enters a client deliverable in regulatory consulting, the failure mode is not just inaccuracy — it is a wrong deliverable that the client then acts on. For Compliance teams at Management & Risk Consulting firms, the specific risk on this regulation is producing a methodology summary or engagement briefing that misrepresents the CPMI-IOSCO assessment's timeline to a regulator, a board, or a counterparty that will rely on it. The table below maps the risk impact of the failures we identified to the specific work products where Compliance teams in this sector carry the exposure.
| Risk Impact | Count | Affected findings |
|---|---|---|
| Wrong deliverable | 1 | Finding#1 |
When this affects your department
Compliance teams at Management & Risk Consulting firms engage with the PFMI Level 3 assessment series in several distinct workflow modes. The most immediate is client advisory: when a trade repository, CCP, CSD, or payment system is formulating its own response to CPMI-IOSCO assessment findings — whether preparing board-level horizon-scanning materials, updating their internal PFMI compliance programmes, or drafting regulatory engagement correspondence — they engage their external consulting firm's Compliance function to provide authoritative framing of what the assessment process entailed and what the findings imply for ongoing obligations.
In this mode, the consulting firm's Compliance team is the source of record. If they reach for AI to accelerate the first draft of a methodology note describing the assessment, and that draft states the assessment concluded in 2024, the client's regulatory engagement summary carries the wrong date. The consulting firm's Compliance team owns that error.
The second mode is internal: training materials, regulatory horizon briefings circulated to sector teams, and onboarding content for new hires advising in the FMI space. A timeline error embedded in internal training materials about the CPMI-IOSCO Level 3 process propagates through the firm's advisory bench. Junior staff presenting to clients — or to regulators in a regulatory dialogue — will quote the wrong timeline as settled fact, with no awareness that the AI-sourced input truncated the assessment by a year.
The stakes for a Management & Risk Consulting firm advising in international FMI markets are asymmetric: an error on a timeline this specific is both highly verifiable and highly consequential. Regulators and sophisticated FMI clients can and will check. A methodology note that misstates when CPMI-IOSCO completed its engagement with participating FMIs is not a minor caveat — it is a factual inaccuracy in a regulatory context where precision is the entire basis of the consulting relationship.
Remediation requires a corrected submission, a client notification, and an explanation of how the error occurred — the kind of incident that surfaces in client relationship reviews and, in regulated advisory contexts, potentially in the firm's own compliance record.
The findings at a glance
The table below summarises the finding identified on the CPMI-IOSCO Level 3 general business risk assessment, covering the question area, the nature of the AI's error, and the risk category for Compliance teams at Management & Risk Consulting firms in international jurisdictions.
| # | Finding title | Type | Citation ID |
|---|---|---|---|
| 1 | Assessment timeline truncated by one year | Hallucination | RLB-F-INT-BIS-CPMI-IOSCO-PFMI-L3-GENERAL-BUSINESS-RISK-2025-Q005 |
Aggregate impact
The finding on this regulation clusters on process chronology: the AI misstated the end date of the CPMI-IOSCO Level 3 assessment, compressing the timeline from the regulator's confirmed 2023–2025 window to a truncated 2023–2024 framing drawn from secondary commentary. This is not a minor rounding error. The PFMI Level 3 assessment process is defined by discrete, documented phases — data collection from FMIs, follow-up engagement rounds, and a formal findings-sharing stage that ran through April 2025. Misrepresenting which of those phases occurred and when is a substantive factual error, not a stylistic one.
The regulator's own description of the process is the authoritative record; the AI conflated summary commentary about an earlier phase with the complete lifecycle.
For Compliance teams at Management & Risk Consulting firms, the systemic risk is that this class of error is plausible-sounding and hard to catch on review. A 2023–2024 range is internally coherent and consistent with what a reviewer who has not read the primary document would expect — it maps to the widely cited data-collection phase. The error only becomes visible when the reviewer checks the primary source directly, which is precisely the step that is skipped when AI is being used to accelerate first-draft production under time pressure.
In a firm where Compliance teams are producing high volumes of client-facing briefings, the review step most likely to be abbreviated is the one that would catch this.
The concentration of the risk on a single deliverable type — the regulatory engagement summary or methodology note — is both the liability and the control point. This regulation's AI failures do not scatter across multiple workflow types; they land on a specific, high-stakes document format that consulting firms produce for FMI clients navigating their regulatory dialogue with CPMI-IOSCO. That specificity means the risk is manageable with targeted controls, but it also means that a firm without those controls is fully exposed on every such document that incorporates AI-assisted drafting.
What your team should do
The default position for any AI-assisted work product touching PFMI Level 3 assessment process facts should be: primary source verification before the document leaves the team. The November 2025 report is the definitive record of when the assessment ran, how engagement with FMIs was structured, and when the findings-sharing phase concluded. AI tools will often draw on secondary commentary — jurisdiction-specific summaries, industry briefings, news coverage from the data-collection phase — that captures only part of the timeline.
For a single-page methodology note, verifying four or five key dates against the primary document takes minutes; the cost of not doing it is a client submission that misstates facts to a regulator.
On this regulation, AI is reasonably reliable for background orientation — understanding the high-level scope of Principle 15, the general architecture of CPMI-IOSCO assessment methodology, or the categories of FMIs in scope. Where it fails is on the specifics that matter most in a formal deliverable: precise dates, the sequence of engagement phases, the number of FMI respondents, and how findings were validated before publication. These are exactly the parameters a methodology note is expected to nail. Treat AI output on these specifics as a starting-point draft requiring mandatory primary-source verification, not a finished product.
For internal training and horizon-scanning materials, the control is the same but the urgency is amplified by scale: an error in an internal briefing circulates to every advisor who reads it. Build a standing team norm that any AI-assisted draft referencing CPMI-IOSCO assessment timelines must be checked against the BIS publication before internal distribution — not just before client delivery. The practical safeguard is to keep the primary document accessible in your team's reference library and make the check a named step in your AI-assisted drafting workflow, not an afterthought.
How RLB Can Help
RegLeg's published hallucination research is available free as a pre-flight check before your team stakes a compliance position on AI-assisted output. If your Compliance function is using AI tools to accelerate regulatory gap analysis, horizon-scanning, or client advisory work across multiple jurisdictions, the research gives you a concrete view of where those tools have already produced materially wrong answers — wrong entities, inverted obligations, fabricated thresholds — on the very regulations your engagements turn on.
That is a faster and more defensible starting point than internal testing alone, and it costs nothing to run it against your current workflow before a client deliverable goes out.
Where the published research ends, we work with your team directly. For Management & Risk Consulting Compliance functions, the highest-exposure workflows tend to cluster around multi-jurisdictional regulatory mapping, AI-supported regulatory change monitoring, and the use of AI tools to draft client-facing regulatory summaries where an incorrect reading travels far. We can run a structured deep-dive — regulator by regulator, workflow by workflow — that maps your actual AI-supported processes to documented failure modes, flags the specific question types where hallucination rates are highest, and gives you a prioritised picture of where supervisory reliance on AI output needs the tightest controls.
We also work with firms on two further workstreams: a confidential review of your existing AI-use policy against our failure-mode catalogue, with prioritised remediation recommendations framed for your risk appetite and client commitments; and the development of training material and CPD-aligned content your Compliance team can use internally to build consistent, evidence-grounded judgment about where AI tools can be trusted and where they require independent verification. Both are scoped to your function's real operating context — not generic AI governance frameworks repurposed for financial services.