AI Labs · updated 2026-05-29 · methodology v2.1

Finding on 'Q014 Probe' for Claude Sonnet 4.6 with web search ON

RLB Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q014-Sonnet46

What the RLB Specialist Panel found

Finding on 'Q014 Probe' for Claude Sonnet 4.6 with web search ON

Question (paraphrased to protect IP): Is the 2017 CPMI-IOSCO guidance on CCP resilience still the current operative standard, or has it been amended or supplemented by more recent developments?
AI's response: "In May 2026 CPMI-IOSCO published a consultative report (d232) on updated guidance related to initial margin and public disclosures, but this appears targeted at updating margin-related guidance rather than replacing the core resilience framework of d163."
Regulator's text: A May 2026 CPMI-IOSCO consultation addresses simulation tools, initial margin responsiveness, margin model governance, overrides, and public disclosures. Comment deadline: 30 June 2026.
Why the AI went wrong: The model correctly identified that a May 2026 consultative document exists and that it supplements rather than replaces the 2017 resilience guidance. However, the characterisation of the document's scope is narrower than the actual consultation, which covers simulation tools and model governance in addition to margin and disclosures. Both cited URLs resolve to pages that do not confirm the claimed scope — one is a BIS press release, the other an FSB publication on a related but distinct topic.
Cited source(s):
https://www.bis.org/press/p260506.htm — Pretextual
https://www.fsb.org/2022/03/fsb-cpmi-and-iosco-analysis-highlights-need-to-co... — Pretextual

Impact for this audience

The model correctly identified the May 2026 consultation but produced a narrower scope characterisation than the actual document, and added a citation to an FSB 2022 document as supporting context — a temporal and topical mismatch that suggests the model is sourcing supporting citations from a related-topic cluster in its training data rather than from retrieved content. The multi-citation pattern with mixed-vintage sources is a diagnostic signal for citation-generation behaviour.

← Previous finding Finding on 'Q010 Probe' for Claude Sonnet 4.6 with web search ON Next finding → Finding on 'Q017 Probe' for Claude Sonnet 4.6 with web search ON

Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q014-Sonnet46

Plain text Download

RegLeg Specialist Panel (2026). "Finding on 'Q014 Probe' for Claude Sonnet 4.6 with web search ON — AI Labs." Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q014-Sonnet46. RegLegBrief AI Hallucination Research, published 2026-05-29. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-pfmi-2012/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-PFMI-2012-v1-014--sonnet-46-websearch/

APA 7th edition Download

RegLeg Specialist Panel. (2026). Finding on 'Q014 Probe' for Claude Sonnet 4.6 with web search ON [Hallucination finding RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q014-Sonnet46]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-pfmi-2012/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-PFMI-2012-v1-014--sonnet-46-websearch/

Bluebook / OSCOLA (US + UK legal) Download

RegLeg Specialist Panel, Finding on 'Q014 Probe' for Claude Sonnet 4.6 with web search ON [RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q014-Sonnet46], RegLegBrief AI Hallucination Research (May 29, 2026), https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-pfmi-2012/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-PFMI-2012-v1-014--sonnet-46-websearch/.

BibTeX Download

@misc{reglegbrief_RLB_H_INT_BIS_CPMI_IOSCO_PFMI_2012_Q014_Sonnet46,
  author    = {RegLeg Specialist Panel},
  title     = {Finding on 'Q014 Probe' for Claude Sonnet 4.6 with web search ON},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-INT-BIS-CPMI-IOSCO-PFMI-2012-Q014-Sonnet46},
  url       = {https://reglegbrief.com/regulators/j1/int/bis-cpmi/cpmi-iosco-pfmi-2012/whitepaper/finding/INT-BIS-CPMI-INT-001-CPMI-IOSCO-PFMI-2012-v1-014--sonnet-46-websearch/}
}

← Back to case study summary Case study detail →

About Citation IDs →