This case study examines how AI tools perform when Insurance Agents in the United Kingdom ask questions about regulatory obligations under the Financial Conduct Authority's Consumer Duty framework. Across one regulation — Consumer Duty (PS22/9 + PRIN 2A) — two distinct questions were tested, and AI assistants produced materially incorrect answers in both cases. The errors are not minor paraphrasing differences: they involve misstatements of the standard a firm must meet and mischaracterisation of which activities fall within scope.
An Insurance Agent who relies on an AI tool's answer without independently verifying against the FCA's published text risks carrying forward a fundamentally wrong understanding of their obligations.
Insurance Agents in the United Kingdom regularly encounter Consumer Duty questions across a wide range of day-to-day activities. An agent scoping a new distribution arrangement will want to know exactly which activities and customer types fall within the Duty's perimeter. One advising a commercial client on group scheme arrangements needs a precise understanding of what is carved out of scope.
Another drafting a compliance memo on harm-avoidance obligations — or preparing a response to an FCA supervisory query — will need to rely on an accurate reading of what the Duty actually requires before a firm can claim a customer's knowing acceptance of a risk provides any protection. AI tools are an increasingly common first port of call in each of these situations, particularly under time pressure.
The stakes for getting these answers wrong are substantial and operate on two levels. For the Insurance Agent personally, acting on an incorrect understanding of the Consumer Duty's scope or its harm-prevention test can expose the agent to professional regulatory action, undermine the quality of advice given to clients, and — where the agent holds a regulated appointment — put their approved person status at risk. Firms that embed incorrect AI-generated guidance into training materials or internal policy documents face the additional risk of systemic non-compliance that may only surface during an FCA supervisory visit or thematic review.
For clients, the consequences are equally serious. A commercial client wrongly advised that Consumer Duty applies to their group insurance distribution activity may incur unnecessary compliance cost, distort product design decisions, or — conversely — a retail-facing client may be left unprotected because an agent incorrectly understood the firm's obligations around foreseeable harm. In either direction, an Insurance Agent's professional credibility and client relationships depend on the accuracy of the regulatory guidance they provide.
Both findings in this case study concern the same regulation — Consumer Duty (PS22/9 + PRIN 2A) — and both errors follow a similar pattern: AI tools do not simply get the answer wrong, they produce responses that are plausible and internally coherent but diverge from the regulator's actual text in consequential ways. In the first finding, AI tools substituted a multi-condition test of their own construction for the single qualifier the FCA actually set. In the second, AI tools directly contradicted an express statutory carve-out by asserting that the Duty can apply to an activity the rule explicitly excludes.
Neither error would be obvious to a reader who had not independently consulted the FCA's published handbook or policy statement.
The concentration of errors within a single regulation is significant. Consumer Duty is one of the most consequential regulatory frameworks Insurance Agents in the United Kingdom currently navigate, and it is precisely the kind of complex, recently implemented rule that practitioners are most likely to query AI tools about. The FCA's published text is detailed and contains carefully drafted qualifiers and exclusions; AI tools appear to over-generalise the Duty's broad consumer-protection intent and apply it beyond its express boundaries, or to replace specific qualifying language with broader, more expansive conditions.
For an Insurance Agent in active practice, the aggregate implication is direct: on the current evidence, an AI tool is more likely to mislead than to clarify when asked a precise question about Consumer Duty scope or the firm's harm-prevention obligations. A practitioner who uses AI-generated answers as a shortcut — without checking against the FCA handbook or the PS22/9 policy statement — faces a meaningful probability of advising clients, drafting documents, or designing compliance processes on a factually incorrect basis.
2 findings in this case study. Click any to see its full evidence card.
Treat AI tools as a starting point for orientation, not as a source of regulatory authority. When a question touches on Consumer Duty scope, the harm-prevention standard, or any exclusion from the Duty's perimeter, the only reliable reference is the FCA's published handbook text and the PS22/9 policy statement. AI tools can help you frame the right question, identify which rule provisions are relevant, or draft an initial summary — but the substance of any compliance position, client advice, or internal policy must be verified against the regulator's own words before it is relied upon.
This applies even when an AI tool cites a specific handbook URL: the citation may appear credible while the characterisation of that source's content is incorrect.
Maintain a clear audit trail for any AI use that feeds into a client deliverable or compliance document. If you have used an AI tool to draft or research a Consumer Duty memo, note that fact and record the independent verification steps you took. This matters both for professional standards purposes and for your own protection if a client or regulator later questions the basis of the advice.
Never paste AI-generated regulatory quotations or rule summaries into a final document without checking them word-for-word against the source — the errors identified in this case study involve precise qualifying language that looks authoritative in isolation.
There are areas where AI tools add genuine value in an Insurance Agent's workflow. Summarising long background documents that you will then verify independently, generating first-draft questions for a client discovery meeting, drafting non-regulatory copy such as covering letters or explanatory notes, and producing initial checklists for a compliance review are all tasks where AI can save time without creating material risk. The key discipline is keeping AI-assisted research clearly separated from the verified regulatory positions that underpin any professional output.
RegLeg's published hallucination research is available as a free reference for Insurance Agents who want to check a specific regulatory topic before acting on an AI-generated answer. The research covers the FCA's Consumer Duty framework — including the findings documented in this case study — and is updated as further testing is completed.
If you are about to rely on an AI answer about Consumer Duty scope, the harm-prevention standard, or any exclusion from the Duty, checking the published findings takes only a few minutes and can confirm whether the topic is one where AI tools are known to produce incorrect responses.
For firms that employ multiple Insurance Agents working across the same regulatory portfolio, RegLeg offers bespoke regulation deep-dives tailored to your specific product lines and distribution arrangements. These engagements map the Consumer Duty provisions most relevant to your firm's activities against the known failure modes in AI-generated responses, giving your compliance and advisory teams a practical reference framework.
We also develop training materials and CPD-aligned content that walk Insurance Agents through the specific types of errors — such as the expansion of qualifying language or the misstatement of scope exclusions — that are most likely to appear when querying AI tools on FCA regulatory obligations.
If your firm already has an AI-use policy or guidance for staff on consulting AI tools for regulatory research, RegLeg can review that policy confidentially against our failure-mode catalogue. This review identifies gaps between the safeguards your policy assumes and the error patterns we have documented, and produces a plain-language summary of recommended adjustments. The aim throughout is to help Insurance Agents and their firms use AI tools productively while maintaining the professional standards and accuracy that FCA-regulated work requires.