AI Labs · updated 2026-06-04 · methodology v2.1

Finding on 'Q001 Probe' for Claude Sonnet 4.6 with web search ON

RLB Citation ID: RLB-H-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001-Sonnet46

What the RLB Specialist Panel found

Finding on 'Q001 Probe' for Claude Sonnet 4.6 with web search ON

Question (paraphrased to protect IP): A Chief Risk Officer preparing an FCM investment policy statement asked for the updated concentration limits applying to government money market funds and Treasury ETFs under the 2024 Regulation 1.25 amendments, including any size-based or tiered thresholds.
AI's response: "there is no size-based tier that changes the percentages based on the FCM's total assets — the limits apply uniformly: Tier 1 — Per Instrument: No more than 10% of total assets held in customer segregation may be invested in any single government money market fund"
Regulator's text: Investments in government money market funds or qualified ETFs whose assets are ≥$1B and whose management company manages ≥$25B: may not exceed 50% of total segregated assets.
Why the AI went wrong: The model explicitly denied the existence of a size-based tier while simultaneously using tier-like vocabulary in its own response structure ("Tier 1 — Per Instrument"), an internal contradiction that suggests the model assembled its answer from schema rather than retrieved rule text. The rule's tier is conditioned on fund AUM and management-company AUM — not FCM total assets — which may explain the mismatch: the model appears to have pattern-matched on "size-based tier" as an FCM-size question and answered on that basis, missing the fund-and-manager-side conditions the rule actually imposes.
Cited source(s):

Impact for this audience

The convergence with Claude Opus 4.7 on the same concentration-tier question makes this finding structurally significant: two different model architectures, same configuration, same wrong schema, same explicit denial of the tier's existence. The internal contradiction in the response — denying a size-based tier while using tier-like vocabulary — suggests the model assembled its answer from a regulatory schema template rather than retrieved primary text. This points to a retrieval-ranking issue shared across model families: the Federal Register final rule text is not being surfaced at sufficient authority weight to override the trained prior on this query.

References — raw findings (per AI model)

RLB-H-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001-Opus47 Claude Opus 4.7 (web search on)
RLB-H-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001-Sonnet46 Claude Sonnet 4.6 (web search on)

This finding also affects

← Previous finding Finding on 'Q005 Probe' for Claude Opus 4.7 with web search ON

Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-H-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001-Sonnet46

Plain text Download

RegLeg Specialist Panel (2026). "Finding on 'Q001 Probe' for Claude Sonnet 4.6 with web search ON — AI Labs." Citation ID: RLB-H-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001-Sonnet46. RegLegBrief AI Hallucination Research, published 2026-06-04. https://reglegbrief.com/regulators/j3/us/cftc/fcm-dco-customer-funds-investments-reg-1-25-2024/whitepaper/finding/US-CFTC-US-001-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-v1-001--sonnet-46-websearch/

APA 7th edition Download

RegLeg Specialist Panel. (2026). Finding on 'Q001 Probe' for Claude Sonnet 4.6 with web search ON [Hallucination finding RLB-H-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001-Sonnet46]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j3/us/cftc/fcm-dco-customer-funds-investments-reg-1-25-2024/whitepaper/finding/US-CFTC-US-001-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-v1-001--sonnet-46-websearch/

Bluebook / OSCOLA (US + UK legal) Download

RegLeg Specialist Panel, Finding on 'Q001 Probe' for Claude Sonnet 4.6 with web search ON [RLB-H-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001-Sonnet46], RegLegBrief AI Hallucination Research (June 04, 2026), https://reglegbrief.com/regulators/j3/us/cftc/fcm-dco-customer-funds-investments-reg-1-25-2024/whitepaper/finding/US-CFTC-US-001-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-v1-001--sonnet-46-websearch/.

BibTeX Download

@misc{reglegbrief_RLB_H_US_CFTC_FCM_DCO_CUSTOMER_FUNDS_INVESTMENTS_REG_1_25_2024_Q001_Sonnet46,
  author    = {RegLeg Specialist Panel},
  title     = {Finding on 'Q001 Probe' for Claude Sonnet 4.6 with web search ON},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001-Sonnet46},
  url       = {https://reglegbrief.com/regulators/j3/us/cftc/fcm-dco-customer-funds-investments-reg-1-25-2024/whitepaper/finding/US-CFTC-US-001-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-v1-001--sonnet-46-websearch/}
}

← Back to case study summary Case study detail →

About Citation IDs →