Why an ISO 42001 Certificate Doesn't Satisfy Your Insurer's Model-Level Audit Obligation
Should you accept an ISO 42001 certificate as your vendor's AI audit report?
The vendor's certificate proves they have a process. Your examiner will ask about the model.
The Situation
Insurers answer to state insurance departments, which set expectations through the National Association of Insurance Commissioners. The NAIC's model bulletin on AI, now adopted by 24 states and the District of Columbia, says your contracts with AI vendors should give you audit rights, or entitle you to audit reports performed by qualified auditing entities. Your vendors will decide what that means when you reach out, and the answer taking shape is an ISO/IEC 42001 certificate. That certificate comes from an audit of how the vendor manages AI across its business. It says nothing about your model, which sets premiums.
The Exposure
Examiner's questions go far beyond whether your AI vendor has a management system. The bulletin tells insurers to expect that examiners will request validation and testing documentation, evaluations of model drift, and confirmation that third parties met their contractual and regulatory obligations. It also expects you to show how you detect unfair discrimination in your insurance practices. Those are questions about outcomes on your book of business, and a vendor management system certificate doesn’t cover those. Connecticut adopted the bulletin, which you would expect from the state that calls Hartford the insurance capital of the world, and it asks in-state insurers to certify that they have an AI governance program and can produce documentation on request. The vendor certificate needs to go in your files, but the broader documentation behind it has to be your own.
The Judgment Call
Your third-party risk team collects vendor assurance material today, following a standard routine: request the report, confirm the issuer, read the opinion, log the exceptions noted and then file it until renewal. An ISO 42001 certificate is easy to add to that routine, so accept it as evidence that the vendor governs AI inside its own organization. But don't let it satisfy your firm's obligation for models that price a policy, decide whether to bind coverage, or pay a claim. For those, split the ask. From the vendor, you want a model-level audit report on the version you deployed, covering how it was built and tested and signed by a firm independent from the development team. From the contract, you need the access that lets you test outcomes on your own book: the decision logs, a test environment, and notice before the version changes. Insurance routinely obtains this type of assurance opinion on other matters, and actuaries who review models do it under their own similar standard, ASOP No. 56, which requires an actuary to disclose when they rely on a vendor's model they can't fully examine. It's the same declaration your examiner will eventually ask you to make. Whatever carriers accept over the next few renewal cycles will become the market's definition of the standard, so your leverage to get what you need and to shape that standard is highest at renewal, while the contract is being renegotiated. Ask for the model-level audit report then, in writing, and you'll find out quickly whether you can conform to the emerging NAIC requirements.
Risk: Your core platform vendor may refuse, and if they’re your policy administration system provider you have no practical ability to walk, so the ask risks some goodwill you may want to reserve for pricing or service terms negotiation.
Benefit: A model-level audit report on the version you actually run gives you the evidence to hand the examiner accurately describing your book.
This Week’s Action
What to do: Pull every AI assurance artifact your third-party risk team holds for the vendor systems that feed policy pricing, bind coverage or pay your claims, and mark whether each one covers the vendor's management system or the model you're running.
Who to involve: Your head of third-party risk or vendor management to produce the file, and your chief actuary or model risk lead to assess what the coverage actually addresses.
What outcome to achieve: A yes/no for each system on whether you hold evidence of outcome testing from your production version. For every no, note the renewal date to make the ask of the vendor.
Time required: 40 minutes to pull and mark the file, 30 minutes with your actuary or model risk lead.
Artifact
Run this against each AI system impacting policy pricing, binding coverage, or paying a claim.
1. Coverage - Does the report cover the vendor's management system, or the specific model you’re licensing?
→ NO REPORT OR MANAGEMENT SYSTEM ONLY: Stop; nothing in it describes your deployment. Log the gap with the contract renewal date and go to Question 6.
→ SPECIFIC MODEL: Continue to Question 2.
2. Version - Does it name the model version running in your production environment today?
→ YES: Continue to Question 3.
→ NO: Stop; ask the vendor which version was tested and when it was last retested.
3. Population - Was the testing run against your book of business, or against the vendor's reference data?
→ YOUR BOOK: Continue to Question 4.
→ VENDOR DATA: Treat the results as valid for what they are, but now plan your own testing.
4. Outcomes - Does it report both bias testing and model drift testing?
→ BOTH: Continue to Question 5.
→ ONE OR NEITHER: Name the missing test and the date you expect to receive it.
5. Signature - Who signed it, and where do they sit?
→ AN OUTSIDE FIRM: Record the firm, the date and the scope. This is your strongest evidence.
→ THE VENDOR'S INTERNAL AUDIT OR COMPLIANCE FUNCTION, REPORTING OUTSIDE THE PRODUCT ORGANIZATION: Acceptable, and your own testing will need to sit behind it.
→ THE TEAM THAT BUILT OR SELLS THE MODEL: Stop; you're holding the developer's assessment of their own work.
6. Entitlement - Does the contract entitle you to the next report, to the decision logs and test access you need for your own testing, and to the vendor's cooperation in an examination?
→ YES: Confirm the notice period.
→ NO: This is your contract renewal ask.
Any system that stops at Question 1 or 2 gives you nothing to show an examiner. One system like that is a vendor problem, but two or more is a gap in your own program.
If your general counsel hasn't looked at what your vendor contracts actually entitle you to receive, question number 6 is the one to put in front of them for the next renewal.
Working through a specific AI decision right now? Reply to this email and tell me what it is. I read every response and answer directly.
When the stakes exceed your internal capacity:
AI Exposure Diagnostic: A 2-hour strategic evaluation for risk, compliance, and legal leaders to identify your highest-priority governance gaps and deliver a 90-day remediation roadmap.
12-Week Governance Sprint: Translate regulatory requirements into audit-ready policies, control frameworks, and accountability structures.
Fractional Chief AI Officer: Embedded ownership of governance, intake, and board reporting before the function is formalized, with the eventual role scoped and the search supported when it's time to hire.
Reply with "Diagnostic," "Sprint," or "Fractional" to schedule a conversation for next month.
Chris Cook writes Judgment Call weekly for compliance and risk officers navigating AI governance.
Former IBM Vice President and Deputy Chief Auditor. Published in the AI Journal, speaker at Yale.
Chris Cook
Managing Partner & Founder
Blackbox Zero
Forwarded this by a colleague? Subscribe to Judgment Call
