AI Hallucination Research › Regulators › Global standard-setters › INT › BIS-CPMI › CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016 › White paper › Detail › Finding
AI Labs · updated 2026-05-26 · methodology v2.1

Finding on 'Q022 Probe' for Claude Sonnet 4.6 with web search ON

RLB Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46
What the RLB Specialist Panel found

Finding on 'Q022 Probe' for Claude Sonnet 4.6 with web search ON

  • Question (paraphrased to protect IP): Has the CPMI-IOSCO 2016 Cyber Resilience Guidance been revised or updated since its publication, and what is its current status as an international standard?
  • AI's response: > As of the date of this response, the June 2016 CPMI-IOSCO Cyber Guidance remains the operative primary international standard for FMI cyber resilience — it has not been formally revised or replaced.
  • Regulator's text: In May 2026, CPMI-IOSCO published a consultative document for public comment on updated guidance. The 2016 guidance is under active revision as of that date.
  • Why the AI went wrong: The model asserted the unchanged operative status of the 2016 guidance without qualification, despite having web-search tools available. The May 2026 CPMI-IOSCO consultation — a publicly announced BIS press release — was not surfaced or reflected in the response. The model's answer reprises its training-era understanding of the document's status as if it were a statement of current fact, with the phrase "as of the date of this response" adding an unwarranted precision to an outdated claim.
  • Regulator portal (if any cited link is dud): https://www.bis.org
Impact for this audience

Like the analogous Opus 4.7 finding, this result implicates the web-search integration's failure to surface recent regulatory announcements that would override a training-era prior. The phrase 'as of the date of this response' in the model's output is particularly significant: it signals that the model is attempting to hedge on currency but does so without actually checking — suggesting the hedging behaviour is a learned linguistic pattern rather than an operationally grounded check. A retrieval step that actively queries for amendment or consultation activity on named regulatory documents before answering status questions would address this gap directly.

References — raw findings (per AI model)
This finding also affects
← Previous finding Finding on 'Q020 Probe' for Claude Sonnet 4.6 with web search ON
Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46
Plain text Download 
RegLeg Specialist Panel (2026). "Finding on 'Q022 Probe' for Claude Sonnet 4.6 with web search ON — AI Labs." Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46. RegLegBrief AI Hallucination Research, published 2026-05-26. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-cyber-resilience-fmi-2016/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-v1-022--sonnet-46-websearch/
APA 7th edition Download 
RegLeg Specialist Panel. (2026). Finding on 'Q022 Probe' for Claude Sonnet 4.6 with web search ON [Hallucination finding RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-cyber-resilience-fmi-2016/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-v1-022--sonnet-46-websearch/
Bluebook / OSCOLA (US + UK legal) Download 
RegLeg Specialist Panel, Finding on 'Q022 Probe' for Claude Sonnet 4.6 with web search ON [RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46], RegLegBrief AI Hallucination Research (May 26, 2026), https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-cyber-resilience-fmi-2016/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-v1-022--sonnet-46-websearch/.
BibTeX Download 
@misc{reglegbrief_RLB_H_INT_BIS_CPMI_IOSCO_CYBER_RESILIENCE_FMI_2016_Q022_Sonnet46,
  author    = {RegLeg Specialist Panel},
  title     = {Finding on 'Q022 Probe' for Claude Sonnet 4.6 with web search ON},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-Q022-Sonnet46},
  url       = {https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-cyber-resilience-fmi-2016/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-CYBER-RESILIENCE-FMI-2016-v1-022--sonnet-46-websearch/}
}
← Back to case study summary Case study detail →
About Citation IDs →