The Question
Your security team just completed an AI inventory exercise. The output surprised leadership: the organization is using AI in 47 distinct applications. Three were built in-house. The remaining 44 are vendor-provided — AI features embedded in SaaS products you have been using for years, purpose-built AI tools procured by individual business units, and API-based AI capabilities integrated into internally developed applications by your engineering team.
Of those 44 third-party AI deployments, your governance team has formal risk assessments for six. The others were procured under standard vendor contracts that cover data handling and security controls. None of the contracts include provisions covering model bias testing, hallucination rates, training data provenance, model update notification, or the EU AI Act "deployer" obligations that now apply to your organization for any high-risk AI systems in that portfolio.
The gap is not unusual. Enterprise vendor risk management frameworks were designed for software, infrastructure, and services. They were not designed for AI systems, where the risk profile is fundamentally different: the behavior of the system is a function of model training, not just code execution, and the behavior can change with model updates that are not surfaced to the customer as software release notes.
Your AI governance program is only as complete as its coverage of AI systems you didn't build — and the vendors selling you those systems are not volunteering the governance information you need.
Why This Matters Now
The EU AI Act's treatment of third-party AI governance is the most significant regulatory development for enterprise procurement teams in this area. The Act distinguishes between "providers" — organizations that develop and place AI systems on the market — and "deployers" — organizations that use AI systems in a professional context. For high-risk AI systems, the Act places substantial obligations on deployers, not just providers.
A deployer of a high-risk AI system must implement a human oversight plan, conduct a fundamental rights impact assessment in applicable contexts, monitor the AI system for performance in production, keep logs of operation where technically feasible, report serious incidents to the provider and, in some cases, directly to national authorities. Critically, the deployer cannot satisfy these obligations by pointing to the provider's documentation — the deployer has its own independent obligations.
This matters for enterprise procurement because virtually every organization deploying Microsoft Copilot, Salesforce Einstein, ServiceNow AI, Workday AI, or any of the rapidly expanding category of AI-powered SaaS products is a "deployer" under the EU AI Act if those systems are used in high-risk decision contexts. The provider — Microsoft, Salesforce, ServiceNow — has its own obligations. But the deployer has separate, independent obligations that the provider's documentation does not satisfy.
In the United States, CFPB guidance, EEOC guidance, and state-level AI regulations similarly place obligations on the organization using the AI, not only the organization that built it. A bank using a third-party credit scoring AI that produces biased outcomes does not have a regulatory defense based on the fact that it purchased the model rather than built it.
Several high-profile vendor incidents in 2024 and 2025 — including undisclosed model updates by major AI API providers that changed output behavior in production applications, and SaaS vendors adding generative AI features to existing products without separate disclosure — have sharpened enterprise awareness of the governance gaps in third-party AI management.
What the CURVE™ Data Shows
The 2026 Stackcurve AI Governance CURVE™ Report evaluated vendors in the AI governance and vendor risk management space for their coverage of third-party AI assessment capabilities. The evaluation assessed purpose-built AI governance platforms, TPRM (third-party risk management) platform vendors that have added AI-specific modules, and CASB/SSPM vendors providing AI application discovery capabilities.
Credo AI ranked as a Leader for third-party AI governance workflows, including a vendor AI assessment questionnaire framework and compliance mapping for EU AI Act deployer obligations. OneTrust has extended its privacy and vendor risk management platform with AI-specific risk assessment templates, with particular strength for organizations that already use OneTrust for GDPR and CCPA compliance. ProcessUnity and Prevalent — established TPRM platforms — both released AI-specific vendor assessment modules in 2024 and 2025, though coverage depth varies. For AI application discovery (identifying what AI tools employees are actually using), Netskope, Zscaler, and Microsoft Defender for Cloud Apps all provide CASB-based AI application inventory capabilities that feed the third-party governance process.
For enterprises that need contractual template guidance, the EU AI Act has generated a body of model contract language for provider-deployer agreements that governance and legal teams are adapting into vendor questionnaire and contract requirements.
The full vendor rankings are in the 2026 Stackcurve AI Governance CURVE™ Report — free to download.
The Gap Most Buyers Miss
Standard vendor risk frameworks do not cover AI-specific risks
SOC 2 Type II, ISO 27001, and standard information security questionnaires cover data security, availability, and access controls. They do not cover the risks specific to AI systems: model bias in outputs, hallucination rates for relevant tasks, training data provenance (was your data or your users' data used to train the model?), model update policies and notification procedures, or the fundamental rights impact of AI-assisted decisions.
A vendor that passes your standard security review with a clean SOC 2 report can simultaneously be deploying a model with significant bias in employment-relevant tasks, using your production data for training without explicit consent, and updating the underlying model with no notification process. The security review did not and cannot surface these risks.
Third-party AI governance requires a separate AI-specific assessment layer, either as a standalone questionnaire or as AI-specific questions added to your standard TPRM process for any vendor deploying AI in your environment.
The AI-embedded-in-SaaS discovery problem
The most common source of unmanaged third-party AI is not a new AI tool procurement — it is an existing SaaS vendor that added AI features to a product already in your environment. Microsoft added Copilot to M365. Salesforce added Einstein GPT to Sales Cloud. Workday added AI-assisted job description generation. Zoom added AI meeting summaries. None of these additions required a new procurement decision, so none of them triggered your TPRM process.
Governance programs need a mechanism for ongoing discovery of AI features added to existing vendor products, not just point-in-time assessments at initial procurement. This requires either contractual notification obligations (vendors must notify you when AI features are added to products in your environment) or continuous discovery tooling that identifies new AI capabilities in use.
The training data question enterprises are not asking
The question that most enterprises are not including in their vendor AI assessments: does the vendor use data submitted through the product — prompts, inputs, user data — to train or fine-tune future model versions? The answer varies significantly by vendor and often by contract tier. OpenAI's API terms (as of 2024) do not use API inputs for training by default. Some SaaS products with embedded AI features use interaction data for model improvement without clear disclosure.
This matters for several reasons: it is a data processing activity that may require disclosure under GDPR and similar privacy regulations; it creates potential confidentiality issues if proprietary business information submitted to an AI feature is incorporated into model training; and it is relevant to the EU AI Act's training data transparency requirements for high-risk AI systems.
Contractual gaps in third-party AI agreements
Standard SaaS agreements do not include provisions for AI-specific requirements. Governance teams reviewing third-party AI contracts should assess whether agreements include: model update notification with reasonable advance notice and rollback rights for production applications; accuracy and reliability SLAs with defined measurement methodology; data processing restrictions that explicitly cover AI inference and training; audit rights for AI-specific assessments; incident notification requirements for model-related failures; and indemnification provisions covering AI-generated outputs that cause customer or regulatory harm.
Most existing contracts are silent on all of these. The remediation path is contract amendment at renewal or renegotiation where AI-related risk warrants it.
Questions Your Buying Team Should Be Asking
1. What training data was used to build the models powering this product, and was any data from customers in our industry segment or comparable use cases included?
Training data provenance affects bias risk, competitive confidentiality, and EU AI Act documentation requirements for high-risk AI systems. Vendors should be able to answer this question at a general level. Inability or unwillingness to answer is itself a governance signal.
2. Does the vendor use data we submit through the product — prompts, inputs, employee or customer data — to train or improve future model versions, and is that covered by our existing DPA?
This question should be asked of every vendor with AI features in your environment, including existing SaaS vendors that have recently added AI capabilities. The answer determines whether your data processing agreements need to be updated and whether employee or customer disclosures need to be revised.
3. What is the model update notification process, and do we have contractual rights to advance notice and rollback before updates go live in our production environment?
Model updates can change output behavior in ways that affect regulated decisions, customer-facing communications, or security posture. Production AI governance requires knowing when the underlying model changes and having the ability to test and approve updates before they affect your operations.
4. Under the EU AI Act, does the vendor classify any AI systems in this product as high-risk, and what documentation do they provide to support our deployer obligations?
For organizations with EU operations or EU-resident users, this question directly surfaces EU AI Act compliance gaps. The vendor's answer — including whether they have assessed their own systems for high-risk classification — is important input for your own deployer obligation assessment.
5. What contractual remedies are available if AI-generated outputs cause customer harm or regulatory non-compliance, and does the vendor's liability cap cover AI-specific incidents?
Most SaaS contracts cap liability at fees paid in the prior 12 months. If a vendor's AI feature generates incorrect regulatory guidance delivered to your customers, the exposure may exceed that cap by orders of magnitude. Understanding where contractual liability sits before an incident is fundamental vendor risk management.
The Stackcurve Take
Third-party AI governance is not a new category of vendor risk management — it is a new layer of requirements on top of existing vendor risk management, triggered by the specific characteristics of AI systems that standard frameworks were not built to assess. Organizations that build AI-specific assessment overlays into their existing TPRM processes — rather than treating third-party AI as a separate program — will get to scale faster and with less organizational friction.
The practical starting point is an AI inventory: a complete picture of which vendors in your environment are using AI to produce outputs that influence decisions, communications, or operations in your organization. Without the inventory, governance is reactive by definition. With the inventory mapped to decision types and regulatory contexts, the priority order for AI-specific assessments becomes clear.
For most enterprises, the highest-priority third-party AI governance work is not exotic new AI tools — it is the embedded AI features in the SaaS products the organization has been using for years, that are now making consequential decisions with no governance overlay.
The 2026 Stackcurve AI Governance CURVE™ Report covers third-party AI governance platforms, AI-specific vendor assessment frameworks, and EU AI Act deployer obligation compliance. Download it free →
Stackcurve Advisory Briefs are independent research. No vendor pays for placement, tier assignment, or editorial influence. The CURVE™ methodology is disclosed in full at stackcurve.net/research/methodology.