AI Hallucination ResearchAudiencesPractitionersInternational / MultilateralLawyersDetail › Finding
Practitioners — Lawyers · updated 2026-06-04
Share / Print Twitter LinkedIn Email

Finding#2 — MGR retroactivity default inverted

RLB Citation ID: RLB-F-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003
AI's failure:Exposed Fabrication Risk for Lawyers:Liability / PI exposure
What the RLB Specialist Panel found
For Claude Opus 4.7 (web search on)
Question (paraphrased to protect IP)

Does the BBNJ Agreement apply to samples of marine genetic resources collected from the high seas before the agreement entered into force?

RLB's analysis

The model stated the opposite of the Agreement's default rule. The Agreement is prospective by design — it covers only post-entry-into-force collections — but the model described a retroactive regime with an opt-out, which inverts both the default and the opt-out structure. This appears to reflect the model reconstructing from commentary on earlier drafts of the text, where the retroactivity question was actively contested, rather than reading the final adopted provision.

AI Head's analysis — what weakness in the AI model caused this

This finding implicates the training data layer: the model appears to have learned the retroactivity rule from pre-adoption negotiating commentary that described an earlier draft rather than the final adopted text. The retrieval step did not correct this because the cited secondary source itself may contain the same error. Both training corpus curation and retrieval-source ranking need to weight post-adoption primary text over pre-adoption commentary for recently adopted instruments.

Cited source(s)
  • https://www.globalpolicywatch.com/2026/03/navigating-the-new-un-high-seas-tre... — Pretextual
For Claude Sonnet 4.6 (web search on)
Question (paraphrased to protect IP)

Does the BBNJ Agreement apply to marine genetic resources collected before the agreement entered into force, or does it operate prospectively only?

RLB's analysis

The model inverted the Agreement's default rule and also inverted the opt-out structure. The Agreement is prospective by default, with parties able to declare retroactive application if they choose. The model described the opposite: retroactive as default, with an opt-out. This is the same fundamental error observed in Claude Opus 4.7 with web search on the same question, which strongly suggests both models are drawing on commentary describing an earlier draft regime rather than the final adopted text.

AI Head's analysis — what weakness in the AI model caused this

This finding, alongside the Opus 4.7 retroactivity finding, strongly suggests a shared training-data origin: both models inverted the same rule in the same direction. The implication for your team is that the error is not a model-specific calibration problem — it is a corpus-level issue that will persist across model versions until the training data for this instrument is corrected. Targeted correction pairs anchored to the final adopted Article 10(1) text are likely the most efficient fix.

Cited source(s)
  • https://www.insideeulifesciences.com/2026/03/03/navigating-the-new-un-high-se... — Pretextual
Impact for Lawyers in international jurisdictions advising on the BBNJ High Seas Biodiversity Agreement

This is the highest-stakes error in the cell. Two independent AI tools asserted that the BBNJ Agreement's marine genetic resource and digital sequence information benefit-sharing rules apply retroactively by default — with a written opt-out available — when Article 10(1) establishes precisely the opposite: non-retroactivity is the default, and provisions apply only to resources collected after entry into force for each Party. A lawyer advising a bioprospecting company, research consortium, or State Party on the legal status of pre-entry-into-force sample collections would produce a fundamentally wrong compliance analysis if this AI response were accepted.

Contracts, access agreements, and benefit-sharing arrangements structured on this error would misallocate obligation and risk across parties.

References — raw findings (per AI model)
This finding also affects
← Previous finding Finding#1 — EIA screening threshold and article misattribution Next finding → Finding#3 — DSI benefit-sharing article misidentified
Cite this finding

Each finding has a stable Citation ID (RLB-F-… for aggregated case-study findings, RLB-H-… for raw per-model hallucinations) — like a DOI, the ID always resolves to the canonical finding even if URLs change.

RLB Citation ID: RLB-F-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003
Bluebook / OSCOLA (US + UK legal) Download
RegLeg Specialist Panel, Finding#2 — MGR retroactivity default inverted [RLB-F-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003], RegLegBrief AI Hallucination Research (June 04, 2026), https://reglegbrief.com/regulators/j1/int/untc/bbnj-high-seas-biodiversity-agreement-2023/practitioners/lawyers/finding/INT-UNTC-INT-001-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-v1-003/.
Plain text Download
RegLeg Specialist Panel (2026). "Finding#2 — MGR retroactivity default inverted — Practitioners — Lawyers." Citation ID: RLB-F-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003. RegLegBrief AI Hallucination Research, published 2026-06-04. https://reglegbrief.com/regulators/j1/int/untc/bbnj-high-seas-biodiversity-agreement-2023/practitioners/lawyers/finding/INT-UNTC-INT-001-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-v1-003/
APA 7th edition Download
RegLeg Specialist Panel. (2026). Finding#2 — MGR retroactivity default inverted [Hallucination finding RLB-F-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003]. RegLegBrief AI Hallucination Research. https://reglegbrief.com/regulators/j1/int/untc/bbnj-high-seas-biodiversity-agreement-2023/practitioners/lawyers/finding/INT-UNTC-INT-001-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-v1-003/
BibTeX Download
@misc{reglegbrief_RLB_F_INT_UNTC_BBNJ_HIGH_SEAS_BIODIVERSITY_AGREEMENT_2023_Q003,
  author    = {RegLeg Specialist Panel},
  title     = {Finding#2 — MGR retroactivity default inverted},
  year      = {2026},
  publisher = {RegLegBrief AI Hallucination Research},
  note      = {Hallucination finding Citation ID: RLB-F-INT-UNTC-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-Q003},
  url       = {https://reglegbrief.com/regulators/j1/int/untc/bbnj-high-seas-biodiversity-agreement-2023/practitioners/lawyers/finding/INT-UNTC-INT-001-BBNJ-HIGH-SEAS-BIODIVERSITY-AGREEMENT-2023-v1-003/}
}
← Back to case study summary Case study detail →