AI Vendor Assessment
Most enterprises use AI capabilities from external vendors — API providers, SaaS platforms, foundation model providers, and specialised AI tools. Each external dependency is a risk vector: the vendor's security posture, data handling practices, model behaviour, and financial stability all affect the risk profile of your AI deployments. AI vendor assessment is more complex than standard IT vendor risk assessment because AI vendors introduce unique risks: data used for model training, unpredictable model behaviour, and AI-specific compliance obligations.
AI Vendor Due Diligence Questionnaire
A structured due diligence questionnaire should cover six domains for any AI vendor:
| Domain | Key questions |
|---|---|
| Data handling | Is customer data used to train models? What data is retained and for how long? Can customers request deletion? Where is data stored and processed geographically? |
| Security | What certifications do they hold (SOC 2 Type II, ISO 27001)? What is the penetration test cadence? How are API keys and credentials managed? What is the incident response SLA? |
| Model transparency | Is a model card or equivalent documentation available? What training data was used? What evaluation was performed for safety and bias? What are known failure modes and limitations? |
| Compliance | Do they sign a DPA (Data Processing Agreement)? Are they GDPR/CCPA compliant? What is their position on EU AI Act obligations? Do they have any regulatory sanctions or pending investigations? |
| Reliability | What is the published and achieved uptime SLA? What is the degradation behaviour when rate limits or quotas are hit? What is the change management process for model updates? |
| Business continuity | What is the vendor's financial runway? Is there a disaster recovery plan? What happens to customer data and access if the vendor ceases operations? Are there escrow arrangements for critical dependencies? |
Certifications to Require
Essential certifications
- SOC 2 Type II: Audited security, availability, processing integrity, confidentiality, and privacy controls over a defined period (minimum 6 months). Type I is a point-in-time assertion — insufficient for high-risk vendors.
- ISO 27001: Certifiable information security management system. Third-party audited. Particularly relevant for EU-regulated deployments.
- ISO 42001: AI-specific management system standard (2023). Early adopters — signals vendor maturity in AI governance.
Sector-specific additions
- Healthcare: HIPAA BAA (Business Associate Agreement) signed; HITRUST CSF certification; FDA compliance for SaMD-relevant AI tools
- Financial services: SOC 1 Type II for financial reporting controls; relevant national banking regulator guidance compliance
- Government / public sector: FedRAMP (US); Cyber Essentials / NCSC alignment (UK); government cloud compliance frameworks
Data Handling Risk Assessment
The highest-risk category of AI vendor risk is data handling — specifically whether your data is used to improve the vendor's models:
Questions that must have confirmed written answers
- Is our input data used to train, fine-tune, or improve the vendor's models? (Many vendors default to yes unless explicitly opted out)
- Can outputs generated from our data be used to train the model? (Even if inputs are excluded)
- What is the data retention period for inputs, outputs, and logs? Is deletion of specific records possible?
- If a breach occurs, how are we notified and within what timeframe?
- Is there sub-processing — does the vendor pass our data to additional third parties (sub-processors)?
Ongoing Monitoring
Vendor risk assessment is not a one-time exercise. AI vendors update models, change data practices, and face financial uncertainty. Approved vendors should be re-assessed on a defined cadence:
| Vendor tier | Criteria | Review cadence |
|---|---|---|
| Critical | Processing sensitive/personal data at scale; embedded in core business processes; no viable fallback | Annual full assessment; quarterly risk review; continuous certification monitoring |
| High | Processing personal data; significant business dependency; alternatives available | Annual full assessment; semi-annual risk review |
| Standard | Non-personal data; non-critical use cases; low switching cost | Biennial assessment; annual confirmation no material changes |
Checklist: Do You Understand This?
- Name the six domains of an AI vendor due diligence questionnaire and one key question for each.
- What is the difference between SOC 2 Type I and Type II, and which should you require?
- What data handling questions must be answered before using any AI vendor that processes personal data?
- What distinguishes a Critical-tier vendor from a Standard-tier vendor in ongoing monitoring?
- Why is vendor assessment not a one-time exercise for AI vendors specifically?