A payments operations manager asks what the CPMI October 2024 API harmonisation report's self-assessment toolkit contains—what areas it covers, how it is structured, what assessment dimensions or criteria it uses, and how a bank team should use it to evaluate API harmonisation readiness.
The model reconstructed a plausible toolkit structure from domain priors and the report's publicly visible scaffold — category names, publication framing — while correctly acknowledging it could not verify the exact question count. The first half of the response commits to specific structural claims (recommendation-by-recommendation workbook, dual operator/participant scope) without retrieved basis; the hedge applies only to the numeric detail, leaving the fabricated structure unqualified.
The model committed to a structural claim (recommendation-keyed workbook, dual operator/participant scope) while hedging only on the numeric count — revealing that the calibration signal for 'I am inferring structure vs. retrieving content' is not uniformly applied. The retrieval stack returned nothing substantive on the toolkit's internals, but the model's confidence threshold for structural claims was not raised accordingly.
A payments operations manager asked what the CPMI d224 self-assessment toolkit contains — its areas, structure, and assessment criteria. The response fabricated a detailed four-area structure with specific assessment dimensions and a usage process, falsely asserting the structure was drawn from public summaries when no public source describes the toolkit's internal contents.
The model did not hedge — it generated a four-area internal structure with named assessment dimensions and a usage process, then falsely attributed this construction to "public summaries." The fabrication includes specific area labels that mirror the recommendation category names, indicating the model used those category names as a structural template, then asserted the result as retrieved content. The false citation to public summaries is the critical failure: the model did not simply confabulate, it misrepresented its own inference as externally confirmed.
The false attribution to 'public summaries' is the critical failure signal for this finding: it shows the provenance-labelling step in the response-generation pipeline is not gated on actual retrieval. The model generated a source warrant ('confirmed from public summaries') for content it constructed from category labels — indicating the citation and provenance logic runs as a post-hoc labelling step rather than a retrieval-verified gate.
A product team at a SaaS firm building an API harmonisation readiness feature, or scoping a self-assessment workflow for a correspondent banking client, would design that feature around toolkit dimensions that do not exist in the actual CPMI document. The error would survive internal sprint review — the fabricated structure is coherent enough to pass non-expert sign-off — and surface only when a technically informed client or bank counterpart compares the product against the real toolkit. Remediation at that stage means redesign, client-facing correction, and a credibility deficit in a market where regulatory precision is a competitive differentiator.
Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.
RegLeg Specialist Panel (2026). "Finding#1 — Fabricated toolkit structure and assessment dimensions — Software Saas × Product Bizdev — International / Multilateral." Citation ID: RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005. RegLegBrief AI Hallucination Research, published 2026-06-04. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/sectors/software_saas/product_bizdev/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-005/
RegLeg Specialist Panel. (2026). Finding#1 — Fabricated toolkit structure and assessment dimensions [Hallucination finding RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/sectors/software_saas/product_bizdev/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-005/
RegLeg Specialist Panel, Finding#1 — Fabricated toolkit structure and assessment dimensions [RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005], RegLegBrief AI Hallucination Research (June 04, 2026), https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/sectors/software_saas/product_bizdev/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-005/.
@misc{reglegbrief_RLB_F_INT_BIS_CPMI_API_HARMONISATION_CROSS_BORDER_2024_Q005,
author = {RegLeg Specialist Panel},
title = {Finding#1 — Fabricated toolkit structure and assessment dimensions},
year = {2026},
publisher = {RegLegBrief AI Hallucination Research},
note = {Hallucination finding Citation ID: RLB-F-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005},
url = {https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/sectors/software_saas/product_bizdev/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-005/}
}