Section 01 · Definition
What AI vendor due diligence actually means
The demo is the one part of the engagement the vendor fully controls, which makes it the least useful signal you have. Due diligence is the work of looking past it.
Quick answer
What is AI vendor due diligence? AI vendor due diligence is the buyer side review you run before signing an AI vendor: you check their technical capability, data handling, model transparency, security record, and true cost, so a polished demo does not hide a production risk you inherit later.
Most AI buying decisions are made on the demo. The vendor shows a clean workflow, the output looks impressive, and the contract follows a week later. The demo is the one part of the engagement the vendor fully controls, which makes it the least useful signal you have. Due diligence is the work of looking past it.
The risk you are pricing is not whether the product works in the demo. It is what you inherit when it runs against your data, your users, and your compliance obligations for the next two years. That risk lives in places the demo never shows: how the vendor handles your data, whether they can explain a model decision when a regulator or a customer asks, what happens to your data when you leave, and whether the price you signed is the price you pay at scale.
A buyer checklist exists to make that invisible risk visible before you commit. You are not trying to catch a vendor lying. You are trying to surface the things a good vendor will answer cleanly and a weak one will dodge. The dodge is the signal.
Section 02 · The checklist
The five areas a buyer checklist must cover
Score each area separately. A vendor can be strong on capability and weak on data handling, and the weak area is the one that hurts you.
Most of the risk in an AI vendor falls into five areas. Score each one on its own. A vendor can be strong on capability and weak on data handling, and the weak area is the one that hurts you.
Technical capability and MLOps maturity
Can they build it, and can they run it after they build it. Capability is table stakes; the maturity to monitor, retrain, roll back, and version models in production is what separates a vendor who ships from one who ships and disappears. Ask how they detect model drift and what their rollback looks like.
Data security and privacy
Where your data lives, who can see it, how it is isolated from other customers, and whether any of it feeds model training. This is the area with the largest downside and the one buyers under examine, because the questions feel adversarial. Ask them anyway.
Model transparency and bias
Whether the vendor can explain how a model reaches a decision, and whether they have tested for biased outcomes on the kind of data you will feed it. A vendor who treats the model as a black box is handing you a liability you cannot defend later.
Security incident history
What has gone wrong before, how they found out, and how they responded. Every vendor with real production exposure has had incidents. The ones worth trusting will tell you about them and show you what changed. Silence here is not a clean record; it is a refusal to answer.
Pricing and total cost of ownership
The headline price is the start. Token costs at your real volume, overage rates, integration and migration effort, and the cost of leaving all belong in the number you compare. A cheap per seat price with punishing usage fees is more expensive than it looks.
Section 03 · The data questions
The three data questions to ask before you sign
If you only had time for three questions, these are the three. Each one has a right answer, and a vendor who cannot give it cleanly is telling you something.
The first is blunt: will our data be used to train your models? The acceptable answers are a contractual no, or an opt in you control with a clear toggle and an audit trail. Anything vaguer, such as we may use aggregated or anonymized data to improve the service, deserves a follow up and a written definition of aggregated, because aggregated often means your data with the obvious identifiers removed and the rest intact.
The second is about isolation: where does our data live, and how is it kept separate from other customers? You want a concrete answer about tenancy, not a reassurance about security in general. A strong vendor explains whether you are in a shared tenant or a dedicated one, what the isolation boundary is, and how they prove it. A weak vendor answers a question about isolation with a sentence about encryption, which is a different control for a different problem.
The third is about the exit: on exit, can we export everything and have you delete it? Data portability and a deletion guarantee belong in the contract, not in a support reply. Ask for the export format and the deletion timeline in writing. A vendor who makes leaving expensive or slow has built a moat out of your data, and you want to see that moat before you are inside it. For the agentic AI specific version of these questions, the agentic AI vendor selection checklist goes deeper on architecture and observability.
Section 04 · Red flags
The red flags that should end the conversation
Most weak answers are a reason to dig further, not to walk away. A few are disqualifying on their own.
Most weak answers are a reason to dig further, not to walk away. A few are disqualifying on their own. These three end the conversation regardless of how good the demo was.
The vendor cannot explain how the model reaches a decision
Not in mathematical detail, but in the practical sense a regulator or a customer will demand: what features drove this output, and how would you defend it. If the answer is the model is proprietary, you are buying a liability you cannot defend when someone challenges an outcome. This is disqualifying for anything that touches a regulated decision.
The vendor cannot prove your data is isolated
If the answer to the isolation question is a brochure about encryption and a refusal to describe the tenancy model, treat your data as commingled until proven otherwise. You cannot fix this after signing, because the architecture is already built. The time to learn your data shares a tenant with a competitor is before the contract, not after a breach.
The vendor will not discuss past incidents
A vendor who claims a spotless record either has no production exposure or is not telling you the truth, and neither is reassuring. The honest version sounds like here is what happened, here is how we found it, and here is what we changed. That answer is a green flag. The refusal to answer is the red one.
If a proposal dodges several of these at once, the patterns in how to evaluate an AI consulting proposal will help you read the rest of it.
Section 05 · Questionnaire
The questionnaire: strong answers versus weak answers
Send the questions in writing and score the answers against what a strong answer looks like. A written questionnaire forces the vendor to commit and gives you a record to compare.
A written questionnaire does two things a meeting cannot. It forces the vendor to commit, and it gives you a record to compare across vendors. The table below is the core of the buyer questionnaire.
| Question to ask | A strong answer | A weak answer |
|---|---|---|
| Will our data train your models? | Contractual no, or an opt in you control | We may use anonymized data to improve the service |
| How is our data isolated? | Named tenancy model with a described boundary | Everything is encrypted and secure |
| Can you explain a model decision? | Feature level reasoning we can defend | The model is proprietary |
| What was your last security incident? | A specific incident with the fix that followed | We have never had one |
| What is the cost at our real volume? | A worked example at our numbers | It depends, but it scales well |
| How do you detect model drift? | A monitoring and retraining process | The model is very accurate |
Score each answer on a simple scale rather than a gut feel. A strong answer is two points, a partial answer is one, a dodge is zero. Add the columns and compare vendors on the total, then look hard at any zero, because a single disqualifying zero outweighs a high score everywhere else.
Section 06 · Operations
The MLOps maturity check most checklists skip
Legal led checklists are thorough on contracts and silent on operations, which is where AI vendors actually fail.
A vendor can pass every privacy clause and still ship a model that quietly degrades three months in because nobody is watching it. MLOps maturity is the operational discipline that keeps a model working after launch, and it is the part of due diligence a procurement template will not cover.
Ask four operational questions. How do you detect when a model is drifting from its training distribution. What does a rollback look like when a new model version is worse than the one it replaced. How often do you retrain, and what triggers it. And who is on call when the model produces a bad output at two in the morning. The answers tell you whether the vendor treats the model as software they operate or as a feature they shipped and forgot.
Grade the lifetime, not the launch
A buyer who scores capability, data handling, and cost but skips operations is grading the launch and ignoring the lifetime. MLOps maturity is what separates a vendor you can rely on for two years from one whose product slowly rots.
If you want a second set of eyes on a shortlist, the agentic AI consulting engagement runs this kind of buyer side technical due diligence, so the questions get asked by someone who has built the thing the vendor is selling.
Section 07 · FAQ
Frequently asked questions
What is AI vendor due diligence?
AI vendor due diligence is the buyer side review you run before signing an AI vendor. You assess their technical capability, data handling, model transparency, security history, and true cost of ownership, so the decision rests on more than a polished demo. The goal is to surface the production risk you would otherwise inherit after the contract is signed.
What should you ask an AI vendor before signing?
Ask the three data questions first: will our data train your models, where does our data live and how is it isolated from other customers, and on exit can we export everything and have it deleted. Then ask whether they can explain a model decision, what their last security incident was, and what the cost is at your real volume. The dodges matter more than the answers.
What are the red flags in an AI vendor?
Three are disqualifying on their own. A vendor who cannot explain how the model reaches a decision, a vendor who cannot prove your data is isolated from other customers, and a vendor who refuses to discuss past security incidents. Each one signals a risk you cannot fix after signing, so treat them as reasons to end the conversation rather than points to negotiate.
Will our data be used to train the vendor's models?
Sometimes, and you have to ask in writing. The acceptable answers are a contractual no or an opt in you control with an audit trail. Be wary of phrasing like aggregated or anonymized data to improve the service, and ask for a written definition, because aggregated often means your data with the obvious identifiers removed and everything else intact.
How do you evaluate an AI vendor's security?
Look past the certifications to the operational answers. Ask how data is isolated between customers, what their last incident was and how they responded, who has access to your data, and what the deletion process looks like on exit. A vendor with real production exposure will describe incidents and fixes plainly. A claim of a perfect record is a refusal to answer, not a clean one.