AI Hallucination ResearchAudiencesPractitionersUnited StatesPublic AuditorsDetail › Finding
Practitioners — Public Auditors · updated 2026-06-04
Share / Print Twitter LinkedIn Email

Finding#1 — Fabricated uniform concentration limits; 50% tiered ceiling omitted

RLB Citation ID: RLB-F-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001
AI's failure:Exposed Fabrication Risk for Public Auditors:Wrong deliverable
What the RLB Specialist Panel found
For Claude Opus 4.7 (web search on)
Question (paraphrased to protect IP)

What are the updated concentration limits that apply to government money market funds and Treasury ETFs under the 2024 Regulation 1.25 amendments, including any tiered or size-based thresholds based on fund and management company asset sizes?

RLB's analysis

The model answered as if the final rule preserved the uniform-threshold structure of prior Regulation 1.25 and rejected size-based differentiation — a description that fits early-stage rulemaking commentary but not the final amended text. The rule introduces an explicit two-condition tier keyed to fund AUM and management company AUM; the model's response overwrites this with a generalised prior-version schema and presents the overwrite as a confirmed regulatory outcome.

AI Head's analysis — what weakness in the AI model caused this

This finding implicates the training corpus's representation of the final rule text relative to pre-final secondary commentary. The model asserted that the final rule rejected size-based tiering — a claim that reflects early rulemaking commentary, not the published final rule. The retrieval pipeline failed to surface the primary rule text at sufficient specificity to override the trained-schema prior, suggesting the Federal Register notice is underweighted relative to secondary sources in retrieval ranking for this query type.

For Claude Sonnet 4.6 (web search on)
Question (paraphrased to protect IP)

A Chief Risk Officer preparing an FCM investment policy statement asked for the updated concentration limits applying to government money market funds and Treasury ETFs under the 2024 Regulation 1.25 amendments, including any size-based or tiered thresholds.

RLB's analysis

The model explicitly denied the existence of a size-based tier while simultaneously using tier-like vocabulary in its own response structure ("Tier 1 — Per Instrument"), an internal contradiction that suggests the model assembled its answer from schema rather than retrieved rule text. The rule's tier is conditioned on fund AUM and management-company AUM — not FCM total assets — which may explain the mismatch: the model appears to have pattern-matched on "size-based tier" as an FCM-size question and answered on that basis, missing the fund-and-manager-side conditions the rule actually imposes.

AI Head's analysis — what weakness in the AI model caused this

The convergence with Claude Opus 4.7 on the same concentration-tier question makes this finding structurally significant: two different model architectures, same configuration, same wrong schema, same explicit denial of the tier's existence. The internal contradiction in the response — denying a size-based tier while using tier-like vocabulary — suggests the model assembled its answer from a regulatory schema template rather than retrieved primary text. This points to a retrieval-ranking issue shared across model families: the Federal Register final rule text is not being surfaced at sufficient authority weight to override the trained prior on this query.

Impact for Public Auditors in the United States advising on the Amendments to Regulation 1.25 — Permissible Investments of Customer Funds by Futures Commission Merchants and Derivatives Clearing Organizations

A public auditor relying on AI output for the concentration-limit framework would build a testing matrix that never tests the ≥$1B fund / ≥$25B management-company two-condition trigger — the entire tiered ceiling disappears, replaced by a flat 10% per-fund assertion the regulation does not contain. An FCM holding a large qualifying government money market fund position above 10% but within the permissible 50% tier would appear non-compliant in the auditor's work papers, or conversely an FCM at 45% with a non-qualifying fund would appear compliant when it is not.

Either direction produces a defective audit conclusion, and the error traces back to parameters the auditor sourced from AI rather than the final rule text.

References — raw findings (per AI model)
This finding also affects
Next finding → Finding#2 — Fabricated SIDR compliance deadline; actual date March 31 2025
Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-F-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001
Plain text Download
RegLeg Specialist Panel (2026). "Finding#1 — Fabricated uniform concentration limits; 50% tiered ceiling omitted — Practitioners — Public Auditors." Citation ID: RLB-F-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001. RegLegBrief AI Hallucination Research, published 2026-06-04. https://reglegbrief.com/regulators/j3/us/cftc/fcm-dco-customer-funds-investments-reg-1-25-2024/practitioners/public-auditors/finding/US-CFTC-US-001-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-v1-001/
APA 7th edition Download
RegLeg Specialist Panel. (2026). Finding#1 — Fabricated uniform concentration limits; 50% tiered ceiling omitted [Hallucination finding RLB-F-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j3/us/cftc/fcm-dco-customer-funds-investments-reg-1-25-2024/practitioners/public-auditors/finding/US-CFTC-US-001-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-v1-001/
Bluebook / OSCOLA (US + UK legal) Download
RegLeg Specialist Panel, Finding#1 — Fabricated uniform concentration limits; 50% tiered ceiling omitted [RLB-F-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001], RegLegBrief AI Hallucination Research (June 04, 2026), https://reglegbrief.com/regulators/j3/us/cftc/fcm-dco-customer-funds-investments-reg-1-25-2024/practitioners/public-auditors/finding/US-CFTC-US-001-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-v1-001/.
BibTeX Download
@misc{reglegbrief_RLB_F_US_CFTC_FCM_DCO_CUSTOMER_FUNDS_INVESTMENTS_REG_1_25_2024_Q001,
  author    = {RegLeg Specialist Panel},
  title     = {Finding#1 — Fabricated uniform concentration limits; 50% tiered ceiling omitted},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-F-US-CFTC-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-Q001},
  url       = {https://reglegbrief.com/regulators/j3/us/cftc/fcm-dco-customer-funds-investments-reg-1-25-2024/practitioners/public-auditors/finding/US-CFTC-US-001-FCM-DCO-CUSTOMER-FUNDS-INVESTMENTS-REG-1-25-2024-v1-001/}
}
← Back to case study summary Case study detail →