The dominant failure observed in Claude Sonnet 4.6 on the CPMI-IOSCO Consultation on Updated Guidance and Public Disclosures to Implement Initial Margin Proposals is deontic register substitution — the model hardened a recommendation into a requirement, replacing 'should' with 'must' when characterising CCP override-framework disclosure obligations. The model identified the correct regulatory subject matter but resolved the consultation's conditional language in the wrong direction, citing a third-party law-firm summary that had editorially strengthened the modal register relative to the primary text. On consultation documents where the normative weight of each modal verb is operationally significant, this failure class carries direct downstream consequences for compliance planning. The failure points to a gap in how the model's retrieval configuration handles deontic precision when secondary-source paraphrase diverges from the regulator's primary text.
This is the consolidated view of findings. Click 'see details →' on any item for the full details for each finding.
This error implicates two subsystems simultaneously: the retrieval ranker's weighting of third-party secondary sources over the regulator's primary text for normative queries, and the model's calibration on deontic precision for consultation-class documents. The training corpus does not appear to carry a strong signal distinguishing 'should' in a consultation from 'must' in a final rule — the model resolved the ambiguity by defaulting to the stronger obligation register, which is the systematically wrong direction for consultation documents.
The Pretextual citation to a law-firm summary rather than the BIS primary text indicates the retrieval pipeline did not prioritise the authoritative source, and the model did not correct for the secondary source's editorial register change.
see details →