AI Hallucination Research › Regulators › Major advanced economies › GB › FCA › CONSUMER-DUTY-PS22-9 › White paper › Detail › Finding
AI Labs · updated 2026-05-26 · methodology v2.1

Finding on 'Q017 Probe' for Claude Opus 4.7 with web search ON

RLB Citation ID: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q017-Opus47
What the RLB Specialist Panel found

Finding on 'Q017 Probe' for Claude Opus 4.7 with web search ON

  • Question (paraphrased to protect IP): Were there significant differences between the Consumer Duty as proposed in CP21/36 and the final rules in PS22/9? What specific provisions were changed?
  • AI's response: > "PS22/9 made several changes from the CP21/36 draft... Key changes/clarifications included: (i) split implementation timeline... (ii) clarifications on scope/distribution chains... (iii) refinement of the foreseeable harm cross-cutting rule wording... (vi) confirmation that a Private Right of Action would NOT be introduced."
  • Regulator's text: The specific differences between CP21/36 and PS22/9 were not independently verifiable from retrieved sources for this evaluation.
  • Why the AI went wrong: The model produced a numbered list of specific changes between the consultation paper and the final policy statement without access to a verified source that documents those changes. Some items may reflect general knowledge of the Consumer Duty policy development process, but the model presented them as a reliable enumerated account without acknowledging that this information was not retrieved from a source. The gap between confident presentation and actual source availability is the substantive failure.
  • Cited source(s):
  • https://www.fca.org.uk/publication/consultation/cp21-36.pdf — Pretextual
  • Regulator portal (if any cited link is dud): https://www.fca.org.uk
Impact for this audience

This finding shows the model constructing a plausible policy history without verified primary sources. The CP21/36-to-PS22/9 comparison requires access to both documents and comparative analysis; the model produced a numbered enumeration without signalling that it was drawing on inference rather than retrieval. This implicates both the retrieval step (which should have attempted to reach the FCA's primary policy documents) and the confidence calibration layer (which should have flagged the absence of verified comparative content).

References — raw findings (per AI model)
← Previous finding Finding on 'Q016 Probe' for Claude Opus 4.7 with web search ON Next finding → Finding on 'Q018 Probe' for Claude Opus 4.7 with web search ON
Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q017-Opus47
Plain text Download 
RegLeg Specialist Panel (2026). "Finding on 'Q017 Probe' for Claude Opus 4.7 with web search ON — AI Labs." Citation ID: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q017-Opus47. RegLegBrief AI Hallucination Research, published 2026-05-26. https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-017--opus-47-websearch/
APA 7th edition Download 
RegLeg Specialist Panel. (2026). Finding on 'Q017 Probe' for Claude Opus 4.7 with web search ON [Hallucination finding RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q017-Opus47]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-017--opus-47-websearch/
Bluebook / OSCOLA (US + UK legal) Download 
RegLeg Specialist Panel, Finding on 'Q017 Probe' for Claude Opus 4.7 with web search ON [RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q017-Opus47], RegLegBrief AI Hallucination Research (May 26, 2026), https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-017--opus-47-websearch/.
BibTeX Download 
@misc{reglegbrief_RLB_H_GB_FCA_CONSUMER_DUTY_PS22_9_Q017_Opus47,
  author    = {RegLeg Specialist Panel},
  title     = {Finding on 'Q017 Probe' for Claude Opus 4.7 with web search ON},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-GB-FCA-CONSUMER-DUTY-PS22-9-Q017-Opus47},
  url       = {https://reglegbrief.com/regulators/j3/gb/fca/consumer-duty-ps22-9/whitepaper/finding/GB-FCA-GB-001-CONSUMER-DUTY-PS22-9-v1-017--opus-47-websearch/}
}
← Back to case study summary Case study detail →
About Citation IDs →