AI Hallucination Research › Regulators › Global standard-setters › INT › BIS-CPMI › CPMI-API-HARMONISATION-CROSS-BORDER-2024 › White paper › Detail › Finding
AI Labs · updated 2026-06-04 · methodology v2.1

Finding on 'Q005 Probe' for Claude Opus 4.7 with web search ON

RLB Citation ID: RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005-Opus47
What the RLB Specialist Panel found

Finding on 'Q005 Probe' for Claude Opus 4.7 with web search ON

  • Question (paraphrased to protect IP): A payments operations manager asks what the CPMI October 2024 API harmonisation report's self-assessment toolkit contains—what areas it covers, how it is structured, what assessment dimensions or criteria it uses, and how a bank team should use it to evaluate API harmonisation readiness.
  • AI's response: "the toolkit is organised as a self-assessment workbook keyed recommendation-by-recommendation, intended for use both by operators of payment systems and by participants/users (banks). I am NOT in a position to verify from memory the exact number of assessment questions"
  • Regulator's text: The landing page states the report is "accompanied by a toolkit for stakeholder self-assessment" and includes "practical implementation actions for each recommendation." No third-party source describes the toolkit's internal contents.
  • Why the AI went wrong: The model reconstructed a plausible toolkit structure from domain priors and the report's publicly visible scaffold — category names, publication framing — while correctly acknowledging it could not verify the exact question count. The first half of the response commits to specific structural claims (recommendation-by-recommendation workbook, dual operator/participant scope) without retrieved basis; the hedge applies only to the numeric detail, leaving the fabricated structure unqualified.
  • Cited source(s):
Impact for this audience

The model committed to a structural claim (recommendation-keyed workbook, dual operator/participant scope) while hedging only on the numeric count — revealing that the calibration signal for 'I am inferring structure vs. retrieving content' is not uniformly applied. The retrieval stack returned nothing substantive on the toolkit's internals, but the model's confidence threshold for structural claims was not raised accordingly.

References — raw findings (per AI model)
This finding also affects
Next finding → Finding on 'Q007 Probe' for Claude Opus 4.7 with web search ON
Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005-Opus47
Plain text Download 
RegLeg Specialist Panel (2026). "Finding on 'Q005 Probe' for Claude Opus 4.7 with web search ON — AI Labs." Citation ID: RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005-Opus47. RegLegBrief AI Hallucination Research, published 2026-06-04. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-005--opus-47-websearch/
APA 7th edition Download 
RegLeg Specialist Panel. (2026). Finding on 'Q005 Probe' for Claude Opus 4.7 with web search ON [Hallucination finding RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005-Opus47]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-005--opus-47-websearch/
Bluebook / OSCOLA (US + UK legal) Download 
RegLeg Specialist Panel, Finding on 'Q005 Probe' for Claude Opus 4.7 with web search ON [RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005-Opus47], RegLegBrief AI Hallucination Research (June 04, 2026), https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-005--opus-47-websearch/.
BibTeX Download 
@misc{reglegbrief_RLB_H_INT_BIS_CPMI_API_HARMONISATION_CROSS_BORDER_2024_Q005_Opus47,
  author    = {RegLeg Specialist Panel},
  title     = {Finding on 'Q005 Probe' for Claude Opus 4.7 with web search ON},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q005-Opus47},
  url       = {https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-005--opus-47-websearch/}
}
← Back to case study summary Case study detail →
About Citation IDs →