AI Hallucination ResearchAudiencesSectorsInternational / MultilateralPayment InstitutionsProduct BizdevDetail › Finding
Payment Institutions × Product Bizdev — International / Multilateral · updated 2026-06-04
Share / Print Twitter LinkedIn Email

Finding#1 — Global FPS count and governance split statistics

RLB Citation ID: RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010
AI's failure:Exposed Fabrication Risk for Payment Institutions × Product Bizdev:Wrong deliverable
What the RLB Specialist Panel found
For Claude Opus 4.7 (web search on)
Question (paraphrased to protect IP)

A market briefing on the global fast payment system landscape needs CPMI data on how many domestic fast payment systems are currently operational globally, how many have already enabled cross-border payment exchanges, how many are planning cross-border linkages, and what proportion are operated by central banks versus private entities.

RLB's analysis

The model substituted figures from a 2025 monitoring survey — a different publication with a different scope and vintage — for the named speech's specific statistics. The ownership breakdown (40% central banks, 35% private entities) from the 2023 speech is absent from the model's response entirely, displaced by the later monitoring survey count. The retrieval pipeline appears to have ranked the more recent survey over the earlier speech, causing the model to answer with the wrong data source for the question asked.

AI Head's analysis — what weakness in the AI model caused this

The substitution of a 2025 monitoring survey count for the correct 2023 speech statistics indicates the retrieval ranker weighted recency over source-match relevance. When the question asks for specific named statistics from a specific named speech, the ranker should surface that speech — not the most recent document that contains related numeric data. The retrieval-routing signal for 'named source in query' vs. 'topic in query' appears not to be differentiated.

For Claude Sonnet 4.6 (web search on)
Question (paraphrased to protect IP)

A market briefing on the global fast payment system landscape was asked to include the proportion of fast payment systems operated by central banks versus private entities. The response correctly cited the 70+ global systems, 14 already cross-border, and 24 planning links — but falsely stated the ownership breakdown was not available in public CPMI sources, when the November 2023 CPMI speech by Tara Rice explicitly gives 40% central bank-operated and 35% privately operated.

RLB's analysis

The model retrieved three statistics from the speech correctly but failed to retrieve the ownership breakdown that appears in the same source. The response asserted the ownership data was "not enumerated" in public sources — a false negative. The retrieval pipeline appears to have indexed the summary-level numeric data from the speech but dropped the ownership-breakdown figures, causing the model to report absence of data that is present in the accessible regulator record.

AI Head's analysis — what weakness in the AI model caused this

The false-negative on ownership breakdown — asserting the 40%/35% figures were 'not enumerated in public sources' when they appear in the same speech the model successfully retrieved other statistics from — points to a partial-retrieval completeness failure. The retrieval pipeline indexed some sentences from the speech but dropped the ownership-breakdown paragraph. The model's self-check did not compare its 'not found' assertion against the full retrieved text of the already-confirmed source.

Cited source(s)
  • https://www.regulationtomorrow.com/eu/cpmi-issues-two-reports-offering-insigh... — Pretextual
Impact for Product & Business Development Teams in Payment Institutions Sector in international jurisdictions working with the Promoting the Harmonisation of Application Programming Interfaces to Enhance Cross-Border Payments: Recommendations and Toolkit

AI tools asked to populate a market briefing with CPMI data on the global fast payment system landscape produced two distinct errors: one substituted the CPMI monitoring survey's respondent count (57 systems) for the authoritative global universe figure of 70+, and another correctly cited headline counts but falsely reported that the central-bank-versus-private-operator breakdown was not publicly available — when the November 2023 CPMI speech by Tara Rice states it explicitly as 40% central bank, 35% private.

For a Product & Business Development team at a Payment Institutions firm, these figures are anchor data in cross-border product pitches, market-entry decisions, and partner briefings; a deliverable carrying the wrong numbers will be challenged by any counterparty who has read the CPMI source material. The credibility damage is compounded by the pretextual citation pattern observed — one AI tool sourced via a secondary summarisation site, meaning a reviewer checking the link would not encounter the correct primary figures.

The control response is straightforward: require direct bis.org source verification for any CPMI ecosystem statistic before it enters an external or board-facing document.

References — raw findings (per AI model)
This finding also affects
Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010
Plain text Download
RegLeg Specialist Panel (2026). "Finding#1 — Global FPS count and governance split statistics — Payment Institutions × Product Bizdev — International / Multilateral." Citation ID: RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010. RegLegBrief AI Hallucination Research, published 2026-06-04. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/sectors/payment_institutions/product_bizdev/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-010/
APA 7th edition Download
RegLeg Specialist Panel. (2026). Finding#1 — Global FPS count and governance split statistics [Hallucination finding RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/sectors/payment_institutions/product_bizdev/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-010/
Bluebook / OSCOLA (US + UK legal) Download
RegLeg Specialist Panel, Finding#1 — Global FPS count and governance split statistics [RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010], RegLegBrief AI Hallucination Research (June 04, 2026), https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/sectors/payment_institutions/product_bizdev/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-010/.
BibTeX Download
@misc{reglegbrief_RLB_F_INT_BIS_CPMI_API_HARMONISATION_CROSS_BORDER_2024_Q010,
  author    = {RegLeg Specialist Panel},
  title     = {Finding#1 — Global FPS count and governance split statistics},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q010},
  url       = {https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/sectors/payment_institutions/product_bizdev/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-010/}
}
← Back to case study summary Case study detail →