AI Hallucination ResearchAudiencesPractitionersInternational / MultilateralFinancial AdvisersDetail › Finding
Practitioners — Financial Advisers · updated 2026-06-04
Share / Print Twitter LinkedIn Email

Finding#1 — Global FPS count and operational-split data misrepresented

RLB Citation ID: RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010
AI's failure:Exposed Fabrication Risk for Financial Advisers:Wrong deliverable
What the RLB Specialist Panel found
For Claude Opus 4.7 (web search on)
Question (paraphrased to protect IP)

A market briefing on the global fast payment system landscape needs CPMI data on how many domestic fast payment systems are currently operational globally, how many have already enabled cross-border payment exchanges, how many are planning cross-border linkages, and what proportion are operated by central banks versus private entities.

RLB's analysis

The model substituted figures from a 2025 monitoring survey — a different publication with a different scope and vintage — for the named speech's specific statistics. The ownership breakdown (40% central banks, 35% private entities) from the 2023 speech is absent from the model's response entirely, displaced by the later monitoring survey count. The retrieval pipeline appears to have ranked the more recent survey over the earlier speech, causing the model to answer with the wrong data source for the question asked.

AI Head's analysis — what weakness in the AI model caused this

The substitution of a 2025 monitoring survey count for the correct 2023 speech statistics indicates the retrieval ranker weighted recency over source-match relevance. When the question asks for specific named statistics from a specific named speech, the ranker should surface that speech — not the most recent document that contains related numeric data. The retrieval-routing signal for 'named source in query' vs. 'topic in query' appears not to be differentiated.

For Claude Sonnet 4.6 (web search on)
Question (paraphrased to protect IP)

A market briefing on the global fast payment system landscape was asked to include the proportion of fast payment systems operated by central banks versus private entities. The response correctly cited the 70+ global systems, 14 already cross-border, and 24 planning links — but falsely stated the ownership breakdown was not available in public CPMI sources, when the November 2023 CPMI speech by Tara Rice explicitly gives 40% central bank-operated and 35% privately operated.

RLB's analysis

The model retrieved three statistics from the speech correctly but failed to retrieve the ownership breakdown that appears in the same source. The response asserted the ownership data was "not enumerated" in public sources — a false negative. The retrieval pipeline appears to have indexed the summary-level numeric data from the speech but dropped the ownership-breakdown figures, causing the model to report absence of data that is present in the accessible regulator record.

AI Head's analysis — what weakness in the AI model caused this

The false-negative on ownership breakdown — asserting the 40%/35% figures were 'not enumerated in public sources' when they appear in the same speech the model successfully retrieved other statistics from — points to a partial-retrieval completeness failure. The retrieval pipeline indexed some sentences from the speech but dropped the ownership-breakdown paragraph. The model's self-check did not compare its 'not found' assertion against the full retrieved text of the already-confirmed source.

Cited source(s)
  • https://www.regulationtomorrow.com/eu/cpmi-issues-two-reports-offering-insigh... — Pretextual
Impact for Financial Advisers in international jurisdictions advising on the Promoting the Harmonisation of Application Programming Interfaces to Enhance Cross-Border Payments: Recommendations and Toolkit

A Financial Adviser who uses AI to populate CPMI benchmark data in a market briefing or client memo receives a figure of 57 operational fast payment systems — roughly 20% lower than the authoritative 70+ figure from the CPMI's own public record. A client acting on that briefing to prioritise jurisdictional connectivity strategy is working from an artificially compressed landscape. Separately, AI tools tested here falsely assert that the central-bank versus private-entity operational split is unquantified in public CPMI sources; the 40%/35% breakdown from the November 2023 Tara Rice CPMI speech is directly on point.

An adviser who accepts that non-answer either leaves the governance-structure question open or wastes time on redundant primary-source research. In either scenario the deliverable is wrong or incomplete, and the adviser has signed off a memo with a stated or implied CPMI data gap that does not exist.

References — raw findings (per AI model)
This finding also affects
Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010
Plain text Download
RegLeg Specialist Panel (2026). "Finding#1 — Global FPS count and operational-split data misrepresented — Practitioners — Financial Advisers." Citation ID: RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010. RegLegBrief AI Hallucination Research, published 2026-06-04. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/practitioners/financial-advisers/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-010/
APA 7th edition Download
RegLeg Specialist Panel. (2026). Finding#1 — Global FPS count and operational-split data misrepresented [Hallucination finding RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/practitioners/financial-advisers/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-010/
Bluebook / OSCOLA (US + UK legal) Download
RegLeg Specialist Panel, Finding#1 — Global FPS count and operational-split data misrepresented [RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010], RegLegBrief AI Hallucination Research (June 04, 2026), https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/practitioners/financial-advisers/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-010/.
BibTeX Download
@misc{reglegbrief_RLB_F_INT_BIS_CPMI_API_HARMONISATION_CROSS_BORDER_2024_Q010,
  author    = {RegLeg Specialist Panel},
  title     = {Finding#1 — Global FPS count and operational-split data misrepresented},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010},
  url       = {https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/practitioners/financial-advisers/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-010/}
}
← Back to case study summary Case study detail →