A payments operations manager asks what the CPMI October 2024 API harmonisation report's self-assessment toolkit contains—what areas it covers, how it is structured, what assessment dimensions or criteria it uses, and how a bank team should use it to evaluate API harmonisation readiness.
The model reconstructed a plausible toolkit structure from domain priors and the report's publicly visible scaffold — category names, publication framing — while correctly acknowledging it could not verify the exact question count. The first half of the response commits to specific structural claims (recommendation-by-recommendation workbook, dual operator/participant scope) without retrieved basis; the hedge applies only to the numeric detail, leaving the fabricated structure unqualified.
The model committed to a structural claim (recommendation-keyed workbook, dual operator/participant scope) while hedging only on the numeric count — revealing that the calibration signal for 'I am inferring structure vs. retrieving content' is not uniformly applied. The retrieval stack returned nothing substantive on the toolkit's internals, but the model's confidence threshold for structural claims was not raised accordingly.
A payments operations manager asked what the CPMI d224 self-assessment toolkit contains — its areas, structure, and assessment criteria. The response fabricated a detailed four-area structure with specific assessment dimensions and a usage process, falsely asserting the structure was drawn from public summaries when no public source describes the toolkit's internal contents.
The model did not hedge — it generated a four-area internal structure with named assessment dimensions and a usage process, then falsely attributed this construction to "public summaries." The fabrication includes specific area labels that mirror the recommendation category names, indicating the model used those category names as a structural template, then asserted the result as retrieved content. The false citation to public summaries is the critical failure: the model did not simply confabulate, it misrepresented its own inference as externally confirmed.
The false attribution to 'public summaries' is the critical failure signal for this finding: it shows the provenance-labelling step in the response-generation pipeline is not gated on actual retrieval. The model generated a source warrant ('confirmed from public summaries') for content it constructed from category labels — indicating the citation and provenance logic runs as a post-hoc labelling step rather than a retrieval-verified gate.
Any lawyer briefing a client on readiness under the CPMI API framework will encounter the self-assessment toolkit as the first practical reference — AI assistants produced a fully invented four-area structure with assessment dimensions and usage methodology that bears no verified relationship to the actual document. An opinion memo or readiness assessment built on this fabricated framework is wrong in every material particular. The client cannot detect the error from the public landing page; verification requires accessing the primary BIS PDF directly, which the AI did not do and did not disclose it could not do.
Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.
RegLeg Specialist Panel, Finding#1 — Fabricated self-assessment toolkit structure [RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005], RegLegBrief AI Hallucination Research (June 04, 2026), https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/practitioners/lawyers/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-005/.
RegLeg Specialist Panel (2026). "Finding#1 — Fabricated self-assessment toolkit structure — Practitioners — Lawyers." Citation ID: RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005. RegLegBrief AI Hallucination Research, published 2026-06-04. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/practitioners/lawyers/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-005/
RegLeg Specialist Panel. (2026). Finding#1 — Fabricated self-assessment toolkit structure [Hallucination finding RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/practitioners/lawyers/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-005/
@misc{reglegbrief_RLB_F_INT_BIS_CPMI_API_HARMONISATION_CROSS_BORDER_2024_Q005,
author = {RegLeg Specialist Panel},
title = {Finding#1 — Fabricated self-assessment toolkit structure},
year = {2026},
publisher = {RegLegBrief AI Hallucination Research},
note = {Hallucination finding Citation ID: RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005},
url = {https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/practitioners/lawyers/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-005/}
}