Executive Summary
The BBNJ Agreement — the United Nations Treaty on Marine Biodiversity of Areas Beyond National Jurisdiction — introduces a binding international framework governing environmental impact assessments, access to marine genetic resources (MGRs), benefit-sharing over digital sequence information (DSI), and area-based management tools in the high seas. For Lawyers practising across international jurisdictions, this treaty creates novel obligations that intersect marine science, intellectual property, environmental law, and the existing architecture of UNCLOS. Across four questions put to AI tools on this regulation, every single one produced a materially incorrect answer.
The errors were not peripheral: they included a direct inversion of the treaty's retroactivity default, a misquotation of the EIA trigger threshold, a wrong article reference for DSI benefit-sharing obligations, and a misattribution of the provision governing the Conference of the Parties' authority over area-based management. In three of the four cases, the AI tool initially answered with confidence and only acknowledged uncertainty when pressed — a pattern that substantially increases reliance risk for any practitioner who does not independently verify every treaty article cited.
How AI gets this regulation wrong
The dominant pattern across AI responses to questions about this treaty is confident fabrication: AI tools stated specific article numbers and operative thresholds with precision, then retracted or qualified those statements when challenged, revealing that their apparent certainty had no reliable foundation. A secondary pattern involves the outright invention of rules — most notably, asserting a treaty default that is the exact opposite of what the text provides. Together, these failure modes make AI assistance on the BBNJ Agreement particularly hazardous for practitioners who need to rely on article-level accuracy to advise clients or draft instruments.
| AI's Failure Mode | Count | Affected findings |
|---|---|---|
| Exposed Fabrication | 3 | Finding#1 · Finding#2 · Finding#3 |
| Misstated Rule | 1 | Finding#4 |
What that means for your practice
Every confirmed error on this regulation carries professional indemnity and liability exposure for Lawyers. Because the BBNJ Agreement governs high-stakes commercial and scientific activities — research expeditions, bioprospecting ventures, shipping route planning, and treaty compliance programmes — incorrect legal advice flowing from AI-generated errors could expose practitioners to negligence claims and expose their clients to regulatory breach. The risk is not theoretical: the errors documented here would each, if taken at face value, produce a fundamentally wrong legal position on treaty obligations that States, international organisations, and private parties are actively beginning to litigate and transact around.
| Risk Impact | Count | Affected findings |
|---|---|---|
| Liability / PI exposure | 4 | Finding#1 · Finding#2 · Finding#3 · Finding#4 |
When this affects Lawyers
Lawyers in international jurisdictions encounter the BBNJ Agreement most frequently when advising clients who conduct or finance marine scientific research in areas beyond national jurisdiction — including academic institutions, biotechnology companies, environmental NGOs, and flag States. Practical questions arise quickly: Does a proposed deep-sea survey require an environmental impact assessment? Which benefit-sharing obligations apply to data derived from biological samples already collected? Does a planned shipping route through a prospective marine protected area remain unaffected by COP decisions? These are precisely the questions that practitioners reach for AI to scope before committing time to primary-source research.
The risk of AI error in this context is amplified by the treaty's novelty. The BBNJ Agreement only opened for signature in 2023 and has not yet accumulated the body of secondary literature, judicial interpretation, and professional commentary that practitioners use to triangulate uncertain AI responses. This means a lawyer who receives a confident AI answer has fewer cross-checks available than they would for a mature instrument, and the AI's apparent confidence is more likely to go unchallenged.
An incorrect EIA threshold or a misidentified article number embedded in a legal opinion or a transactional disclosure document could constitute a material professional error.
The stakes extend beyond individual client advice. Practitioners advising States on treaty implementation, companies on regulatory compliance programmes, or research institutions on data-governance policies may reproduce AI-generated errors at scale — building internal policies, training materials, or contract templates on a fundamentally wrong understanding of the treaty's operative provisions.
The findings at a glance
The table below summarises each confirmed error identified when AI tools were asked questions about the BBNJ Agreement that Lawyers in international jurisdictions are likely to raise in practice.
Aggregate impact
The errors across these four findings are not random: they cluster on the treaty's most commercially and legally consequential provisions — the EIA trigger, the MGR retroactivity default, the DSI benefit-sharing article, and the COP's ABMT authority. All four involve article-level misattributions or substantive rule inversions. What this reveals is a systematic pattern in which AI tools have absorbed enough about the BBNJ Agreement to generate plausible-sounding answers, but not enough to get the operative text right at the level of precision that legal practice requires.
The retroactivity inversion (Finding 2) is the most dangerous error in the set. The BBNJ Agreement is explicitly non-retroactive by default: MGR and DSI provisions apply only to resources collected after the treaty enters into force for each Party. AI tools tested on this point stated the opposite — that the regime is retroactive by default, with an opt-out available. If a lawyer advising a bioprospecting company or a research institution takes that answer at face value, the client's legal position on access agreements and benefit-sharing obligations for existing sample collections would be entirely wrong.
Multiple AI tools produced this error independently, suggesting it is not idiosyncratic to a single model's training data.
The EIA threshold error (Finding 1) compounds this picture: an AI tool replaced the treaty's precautionary "may have more than a minor or transitory effect" standard with the higher-bar formulation "likely to have," narrowing the screening trigger in a way that could lead practitioners to advise that an EIA is not required when it is. Taken together, the errors in this cell represent a near-complete failure of AI reliability on the treaty provisions that Lawyers in international jurisdictions are most likely to need accurate answers on quickly.
What your team should do
The default position for Lawyers advising on the BBNJ Agreement should be: treat AI-generated article citations as unverified until confirmed against the treaty text. The errors documented here are not matters of interpretation — they involve the wrong article number, the wrong operative standard, and the inverted operative default. None of these would be exposed by a plausibility check; they require line-by-line comparison with the United Nations Treaty Collection text published at treaties.un.org. For any work product that will be delivered to a client or used to structure a transaction, independent verification is not optional.
In practical terms, this means establishing a short checklist for any AI-assisted research on the BBNJ Agreement: confirm the article number cited, confirm the operative threshold or standard verbatim, and confirm whether a provision is stated as a default or an opt-in. These three checks would have caught every error in this cell. Firms and chambers with active BBNJ practices should consider building treaty text lookups directly into their research workflow rather than relying on AI to supply article-level precision.
AI tools remain useful for orientation on the BBNJ Agreement — understanding the broad structure, identifying which parts of the treaty address a given topic, and generating first-draft outlines of advice memoranda. The hazard lies in trusting AI to supply precise article numbers, operative thresholds, and default rules without verification. Given that the treaty text is freely and publicly accessible, the cost of verification is low; the cost of propagating an inverted retroactivity rule or a narrowed EIA threshold into client advice is not.
How RLB Can Help
RegLeg's published Hallucination Research is available as a free pre-flight check for lawyers working on regulatory matters. Before relying on AI-assisted output — whether for advice, drafting, or due diligence — lawyers can consult the research to understand which failure modes have been observed for the specific regulation in question. This is not a substitute for legal judgement, but it is a structured, independent reference that flags where AI tools have historically misfired, allowing practitioners to focus their human verification effort on the highest-risk points.
For firms where multiple lawyers work across the same regulatory portfolio, RegLeg offers bespoke deep-dive engagements. These go beyond the published research to examine the specific regulations, jurisdictions, and question types most relevant to the firm's practice. The output is a tailored briefing that legal teams can use as a standing reference — updated as the regulatory landscape evolves — giving the whole team a shared, consistent picture of where AI tools should be treated with caution and where they have performed reliably.
RegLeg also works with legal teams on training and CPD-aligned content. This covers the categories of failure lawyers are most likely to encounter — including outdated regulatory text, cross-jurisdictional confusion, and misattributed citations — framed around real regulatory examples rather than abstract AI theory. Separately, RegLeg can conduct a confidential review of a firm's existing AI-use policy, assessing it against the failure-mode catalogue the research has surfaced. The output is a structured gap analysis: which risks the policy already addresses, which it does not, and where practical amendments would strengthen the firm's position.