AI Labs · updated 2026-05-31 · methodology v2.1

Finding on 'Q012 Probe' for Claude Sonnet 4.6 with web search ON

RLB Citation ID: RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q012-Sonnet46

What the RLB Specialist Panel found

Finding on 'Q012 Probe' for Claude Sonnet 4.6 with web search ON

Question (paraphrased to protect IP): How did the European Parliament vote on BBNJ ratification in April 2024, and what was the subsequent EU ratification procedure?
AI's response: "The European Parliament gave its consent in April 2024 (with the Environment Committee voting on 18 April 2024, followed by the full Parliament Plenary vote approving consent)."
Regulator's text: The EU Parliament gave its consent to BBNJ ratification by an overwhelming majority in April 2024; the EU was committed to depositing its instrument of ratification before the June 2025 UN Ocean Conference.
Why the AI went wrong: The model's answer was substantively correct on the EP vote and the consent procedure. It is included because the model cited the Council of the EU press release on the ratification decision — a document that post-dates the EP consent vote — rather than a primary source for the Parliament vote itself. The procedural detail about the Environment Committee voting on 18 April before the Plenary is consistent with standard EP legislative procedure, but the model presented it as confirmed fact from its search rather than as procedural inference. For questions about specific legislative votes, the model should distinguish between retrieved confirmation and procedural reconstruction from general knowledge.
Cited source(s):
https://www.consilium.europa.eu/en/press/press-releases/2024/06/17/high-seas-... — Pretextual

Impact for this audience

This finding is substantively correct but illustrates a pattern of procedural reconstruction: the model described the EP Environment Committee vote date (18 April 2024) as confirmed from its search when the cited Council source does not directly confirm that date — the model appears to have inferred it from general knowledge of EP legislative procedure. For questions about specific legislative procedural steps, models should distinguish between retrieved confirmation and procedural inference, and flag where the answer is reconstructed rather than directly sourced.

← Previous finding Finding on 'Q011 Probe' for Claude Sonnet 4.6 with web search ON

Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q012-Sonnet46

Plain text Download

RegLeg Specialist Panel (2026). "Finding on 'Q012 Probe' for Claude Sonnet 4.6 with web search ON — AI Labs." Citation ID: RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q012-Sonnet46. RegLegBrief AI Hallucination Research, published 2026-05-31. https://reglegbrief.com/regulators/j1/int/untc/bbnj-high-seas-biodiversity-agreement-2023/whitepaper/finding/INT-UNTC-INT-001-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-v1-012--sonnet-46-websearch/

APA 7th edition Download

RegLeg Specialist Panel. (2026). Finding on 'Q012 Probe' for Claude Sonnet 4.6 with web search ON [Hallucination finding RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q012-Sonnet46]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/untc/bbnj-high-seas-biodiversity-agreement-2023/whitepaper/finding/INT-UNTC-INT-001-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-v1-012--sonnet-46-websearch/

Bluebook / OSCOLA (US + UK legal) Download

RegLeg Specialist Panel, Finding on 'Q012 Probe' for Claude Sonnet 4.6 with web search ON [RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q012-Sonnet46], RegLegBrief AI Hallucination Research (May 31, 2026), https://reglegbrief.com/regulators/j1/int/untc/bbnj-high-seas-biodiversity-agreement-2023/whitepaper/finding/INT-UNTC-INT-001-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-v1-012--sonnet-46-websearch/.

BibTeX Download

@misc{reglegbrief_RLB_H_INT_UNTC_BBNJ_HIGH_SEAS_BIODIVERSITY_AGREEMENT_2023_Q012_Sonnet46,
  author    = {RegLeg Specialist Panel},
  title     = {Finding on 'Q012 Probe' for Claude Sonnet 4.6 with web search ON},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-H-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q012-Sonnet46},
  url       = {https://reglegbrief.com/regulators/j1/int/untc/bbnj-high-seas-biodiversity-agreement-2023/whitepaper/finding/INT-UNTC-INT-001-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-v1-012--sonnet-46-websearch/}
}

← Back to case study summary Case study detail →

About Citation IDs →