AI Hallucination Research › Regulators › Global standard-setters › INT › BIS-CPMI › CPMI-API-HARMONISATION-CROSS-BORDER-2024 › White paper › Detail › Finding
AI Labs · updated 2026-06-04 · methodology v2.1

Finding on 'Q007 Probe' for Claude Sonnet 4.6 with web search ON

RLB Citation ID: RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q007-Sonnet46
What the RLB Specialist Panel found

Finding on 'Q007 Probe' for Claude Sonnet 4.6 with web search ON

  • Question (paraphrased to protect IP): An advisor was asked which central bank CPMI had specifically named as its partner for advancing the payment pre-validation API recommendation under d224. The AI responded that no central bank had been publicly identified in this role and proposed the Bank of England as the closest named involvement — when CPMI Brief No. 9 explicitly names the South African Reserve Bank (SARB) as CPMI's named collaboration partner conducting joint stakeholder interviews on the pre-validation API recommendation.
  • AI's response: "Based on my research, no single central bank is publicly identified in CPMI documents as formally 'partnering' with CPMI on the payment pre-validation API recommendation in a named bilateral arrangement... The Bank of England's involvement is the closest to a named partnership"
  • Regulator's text: CPMI Brief No. 9 (November 2025): "The CPMI, in collaboration with the South African Reserve Bank (SARB), has been advancing the API recommendation on payment pre-validation by conducting interviews with market stakeholders."
  • Why the AI went wrong: The model's retrieval pipeline did not surface CPMI Brief No. 9, which names SARB explicitly. Rather than returning a clean "not found," the model substituted the Bank of England — a higher-frequency institution in CPMI-adjacent content — as the most plausible named partner. The substitution follows a retrieval-ranking pattern where the model fills a retrieval gap with the highest-prior institution, displacing the correct lower-frequency answer.
  • Cited source(s):
Impact for this audience

The Bank of England substitution for SARB is a high-prior-institution fill pattern: the retrieval pipeline did not surface CPMI Brief No. 9, so the model substituted the most contextually plausible high-frequency institution. This failure mode will recur whenever the correct answer is a lower-frequency named institution in a recent sub-document that the retrieval index has not fully covered — a structural property of how institutional attribution fails under sparse indexing.

References — raw findings (per AI model)
This finding also affects
← Previous finding Finding on 'Q005 Probe' for Claude Sonnet 4.6 with web search ON Next finding → Finding on 'Q008 Probe' for Claude Sonnet 4.6 with web search ON
Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q007-Sonnet46
Plain text Download 
RegLeg Specialist Panel (2026). "Finding on 'Q007 Probe' for Claude Sonnet 4.6 with web search ON — AI Labs." Citation ID: RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q007-Sonnet46. RegLegBrief AI Hallucination Research, published 2026-06-04. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-007--sonnet-46-websearch/
APA 7th edition Download 
RegLeg Specialist Panel. (2026). Finding on 'Q007 Probe' for Claude Sonnet 4.6 with web search ON [Hallucination finding RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q007-Sonnet46]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-007--sonnet-46-websearch/
Bluebook / OSCOLA (US + UK legal) Download 
RegLeg Specialist Panel, Finding on 'Q007 Probe' for Claude Sonnet 4.6 with web search ON [RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q007-Sonnet46], RegLegBrief AI Hallucination Research (June 04, 2026), https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-007--sonnet-46-websearch/.
BibTeX Download 
@misc{reglegbrief_RLB_H_INT_BIS_CPMI_API_HARMONISATION_CROSS_BORDER_2024_Q007_Sonnet46,
  author    = {RegLeg Specialist Panel},
  title     = {Finding on 'Q007 Probe' for Claude Sonnet 4.6 with web search ON},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-INT-BIS-CPMI-API-HARMONISATION-CROSS-BORDER-2024-Q007-Sonnet46},
  url       = {https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-api-harmonisation-cross-border-2024/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-API-HARMONISATION-CROSS-BORDER-2024-v1-007--sonnet-46-websearch/}
}
← Back to case study summary Case study detail →
About Citation IDs →