Executive Summary
The BBNJ Agreement — the United Nations treaty on the conservation and sustainable use of marine biological diversity of areas beyond national jurisdiction — establishes, for the first time, a binding international framework governing how marine genetic resources (MGRs) collected from the high seas must be shared, reported, and benefitted from. For Legal teams at biotechnology firms operating across international jurisdictions, the Agreement creates direct compliance obligations around access, disclosure, and benefit-sharing that touch every stage of the research and commercialisation pipeline.
Across two questions our research put to AI tools on this regulation, the AI answered incorrectly in both cases — and, when pressed, acknowledged its own uncertainty rather than standing behind its stated position. In both instances the AI's errors concerned foundational rules of the treaty: whether pre-entry-into-force collections are retroactively covered, and which specific article governs digital sequence information benefit-sharing. These are not peripheral details — they are the legal anchors around which a biotechnology firm's entire BBNJ compliance posture is built.
How AI gets this regulation wrong
The AI failures documented on this regulation share a common shape: the AI gave a confident, plausible-sounding answer, and only when its own reasoning was probed did it concede the answer was uncertain or wrong. Across both findings the errors involved the AI either inverting a core treaty rule — stating the opposite of what the Agreement actually says — or attaching an obligation to the wrong article entirely. These are not ambiguous edge-cases; they are the kind of confident misstatement that, passed without verification into a legal memo or compliance briefing, could anchor an incorrect compliance position.
| AI's Failure Mode | Count | Affected findings |
|---|---|---|
| Exposed Fabrication | 2 | Finding#1 · Finding#2 |
What that means for your team
Both findings in this cell map to the same category of operational risk: the firm acts on a wrong deliverable — a legal analysis, compliance note, or internal briefing that rests on an incorrect statement of law. For a biotechnology firm in international jurisdictions, the consequences are particularly acute because the BBNJ framework is new, fast-moving, and directly conditions whether existing and future marine-derived research programmes are lawful. The table below sets out where in the Legal team's work these risks materialise and what the firm stands to lose.
| Risk Impact | Count | Affected findings |
|---|---|---|
| Wrong deliverable | 2 | Finding#1 · Finding#2 |
When this affects your department
A biotechnology firm's Legal team encounters the BBNJ Agreement at precisely the moments when precision matters most. When the firm is assessing whether heritage collections of marine-derived samples — gathered during earlier research expeditions or obtained through legacy material transfer agreements — are subject to the new treaty's benefit-sharing obligations, the question of retroactivity is not abstract: it determines whether costly compliance programmes must be retrofitted onto existing assets.
Similarly, when the firm is drafting access and benefit-sharing agreements for new programmes, deciding how to disclose the use of digital sequence information, or advising the R&D team on which sample repositories can lawfully be drawn upon, the Legal team will almost certainly turn to AI tools to get a rapid orientation on the treaty text before engaging specialist external counsel.
The risk is compounded by timing. The BBNJ Agreement is newly in force and specialist commentary is still thin. AI tools, trained on pre-ratification material or secondary sources that pre-date the final text, are structurally prone to mischaracterising core provisions — and a Legal team under time pressure may not have the regulatory depth to catch a confident but inverted rule.
If the firm's internal advice incorrectly treats pre-entry-into-force collections as retroactively caught by the treaty (the exact inversion documented in Finding 1), it may commit to unnecessary and costly benefit-sharing disbursements, trigger internal restructuring of its heritage portfolio, or introduce false compliance obligations into licensing negotiations with partners and acquirers.
The stakes are equally high at the other end: if the firm incorrectly concludes that digital sequence information is not caught — or is caught only under a different article than the one that actually applies — its disclosure and reporting processes may be misconfigured from the outset. Third-party due diligence, investor representations, and regulatory filings that reference the wrong treaty provision can expose the firm to reputational harm, renegotiation demands, and adverse regulatory attention as national implementing legislation begins to crystallise.
The findings at a glance
The table below summarises each finding — the question the Legal team might ask, what the treaty actually says, and how the AI answered instead.
Aggregate impact
Taken together, the two findings documented here share a structural pattern that is particularly dangerous for a new treaty: AI tools are systematically misconfiguring the foundational rules rather than getting peripheral details wrong. The first finding inverts the Agreement's default temporal scope — turning a non-retroactive regime into a retroactive one — while the second misattributes the digital sequence information benefit-sharing obligation to the wrong article. Both errors involve provisions that are not obscure sub-clauses; they are load-bearing articles whose correct interpretation anchors everything else the Legal team does on this regulation.
The clustering of errors on Article 10 (retroactivity) and Article 14 (DSI benefit-sharing) is significant. These are the two provisions that biotechnology firms were most keenly watching during the treaty negotiations, precisely because they determine how the firm must handle its most commercially sensitive assets: historical collections and genomic data derived from high-seas organisms. An AI that consistently gets these articles wrong will produce confident, internally consistent legal analyses — often citing secondary sources that themselves misread the treaty — that Legal teams acting in good faith may not immediately flag as wrong.
The systemic risk for firms across international jurisdictions is amplified by the fact that the BBNJ Agreement is implemented through national law, and that implementing legislation is still being drafted in many relevant jurisdictions. A Legal team that forms an incorrect baseline understanding of the treaty's scope now — assisted by AI tools — will carry that misunderstanding into its engagement with domestic implementing rules, potentially embedding the error deeper into the firm's compliance architecture before specialist review catches it.
What your team should do
The default position for Legal teams on the BBNJ Agreement should be that AI tools are not a reliable first source for the operative text of specific articles. The treaty is recent, implementing guidance is sparse, and AI tools trained on pre-ratification commentary or secondary sources will frequently present plausible-sounding paraphrases of treaty provisions that silently invert the actual rule.
For any question that turns on which article applies, what the default rule is, or whether a provision is retroactive or prospective, the team should work directly from the consolidated treaty text available through the United Nations Treaty Collection and flag the specific article before incorporating any AI-sourced summary into a work product.
Where AI tools remain useful for this regulation is in orientation work — building familiarity with the treaty's overall structure, understanding the relationship between the MGR and DSI regimes in general terms, or generating a list of the questions the Legal team should be asking before briefing external specialists. AI can also assist with drafting the non-normative portions of internal briefings and policy documents, provided the operative legal statements are drafted or verified against the treaty text itself.
The firm should establish a clear internal protocol: AI-assisted research on BBNJ must be flagged and the relevant articles cited and checked, rather than treated as a reliable summary.
For the two issues specifically flagged in this cell, the team should confirm its position on Article 10(1) and Article 14(1) directly from the treaty text. Article 10(1) establishes a non-retroactive default: the MGR and DSI provisions apply only to resources collected and generated after the Agreement enters into force for each Party. Article 14(1) is the operative provision for digital sequence information benefit-sharing obligations. Both are straightforward to verify in under ten minutes from the official treaty text — a step that eliminates the risk entirely.
How RLB Can Help
RegLeg's published Hallucination Research gives the Legal team at a Biotechnology firm a ready pre-flight check before placing any reliance on AI-generated output for regulatory questions. Because the research is public and regulation-specific, it can be consulted directly — without any engagement with RLB — to understand which question types have already produced demonstrable AI failures on the rules your team works with most. That alone helps Legal counsels calibrate how much independent verification a given AI response warrants before it informs a submission, a compliance memo, or external advice.
Where a firmer foundation is needed, RLB works with Legal functions to conduct bespoke regulator deep-dives: a structured mapping of the AI-supported workflows already in use — or under consideration — against the failure modes that Hallucination Research has surfaced for the relevant regulatory perimeter. For Biotechnology firms operating across multiple jurisdictions, this typically surfaces material differences in hallucination exposure by product category, approval pathway, and regulator — intelligence that a generic AI risk register will not capture. The output is a prioritised exposure map the Legal team can act on, rather than a general caution about AI reliability.
RLB also offers a confidential review of an existing AI-use policy against our failure-mode catalogue, with prioritised remediation guidance framed around the specific regulatory workflows the policy covers. Findings are delivered in formats the Legal team can use directly: internal briefing notes, decision frameworks for when AI output requires secondary verification, and CPD-aligned training material teams can deploy to keep practitioners current as both the regulatory landscape and the AI tooling evolve.