AI Hallucination Research › Regulators › Major advanced economies › GB › FCA › CONSUMER-DUTY-PS22-9 › White paper › Detail › Finding
AI Labs · updated 2026-05-26 · methodology v2.1

Finding on 'Q002 Probe' for Claude Sonnet 4.6 with web search ON

RLB Citation ID: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46
What the RLB Specialist Panel found

Finding on 'Q002 Probe' for Claude Sonnet 4.6 with web search ON

  • Question (paraphrased to protect IP): Was the Consumer Duty created through primary legislation or FCA rulemaking? What is the legal basis for Principle 12 and PRIN 2A, and did FSMA 2023 have any role in creating it?
  • AI's response: > "The Consumer Duty was introduced by FCA rules, not by an Act of Parliament. The legal basis is the FCA's statutory rule-making power under the Financial Services and Markets Act 2000 (FSMA 2000)."
  • Regulator's text: FSMA 2023 did not create the Consumer Duty.
  • Why the AI went wrong: The model correctly identified that the Consumer Duty is FCA rulemaking under FSMA 2000, but it did not address the role of FSMA 2023, which was specifically raised in the question. The response omits any engagement with whether FSMA 2023 had any role — a question for which the FCA's published record provides a clear answer. By not addressing this element, the model left a material gap that could mislead a reader into thinking FSMA 2023 is simply irrelevant rather than affirmatively excluded.
  • Cited source(s):
  • https://www.akingump.com/en/insights/alerts/new-fca-principle-12-consumer-duty — Pretextual
  • Regulator portal (if any cited link is dud): https://www.fca.org.uk
Impact for this audience

This finding implicates the model's handling of compound regulatory questions where one element has a clear published answer (the general legal basis) and another element requires engagement with a specific recent Act (FSMA 2023). The model answered the first element correctly and silently omitted the second. This is a selective engagement pattern — the model responded to the part of the question it had strong training coverage on and left the novel element unaddressed. Eval probes that include a secondary specific element alongside a general question would test this pattern.

References — raw findings (per AI model)
← Previous finding Finding on 'Q020 Probe' for Claude Opus 4.7 with web search ON Next finding → Finding on 'Q005 Probe' for Claude Sonnet 4.6 with web search ON
Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46
Plain text Download 
RegLeg Specialist Panel (2026). "Finding on 'Q002 Probe' for Claude Sonnet 4.6 with web search ON — AI Labs." Citation ID: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46. RegLegBrief AI Hallucination Research, published 2026-05-26. https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-002--sonnet-46-websearch/
APA 7th edition Download 
RegLeg Specialist Panel. (2026). Finding on 'Q002 Probe' for Claude Sonnet 4.6 with web search ON [Hallucination finding RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-002--sonnet-46-websearch/
Bluebook / OSCOLA (US + UK legal) Download 
RegLeg Specialist Panel, Finding on 'Q002 Probe' for Claude Sonnet 4.6 with web search ON [RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46], RegLegBrief AI Hallucination Research (May 26, 2026), https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-002--sonnet-46-websearch/.
BibTeX Download 
@misc{reglegbrief_RLB_H_GB_FCA_CONSUMER_DUTY_PS22_9_Q002_Sonnet46,
  author    = {RegLeg Specialist Panel},
  title     = {Finding on 'Q002 Probe' for Claude Sonnet 4.6 with web search ON},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q002-Sonnet46},
  url       = {https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-002--sonnet-46-websearch/}
}
← Back to case study summary Case study detail →
About Citation IDs →