The Question
How much of your AI stack did you actually build?
For most enterprises, the honest answer is: very little. The foundation model came from OpenAI, Anthropic, Meta, or Google. The vector database is an open-source project. The orchestration framework is LangChain or LlamaIndex. The fine-tuning datasets were sourced from public repositories. The model files were downloaded from Hugging Face.
Every one of those dependencies is a link in a supply chain. Every link is a potential attack vector. And unlike traditional software supply chain security — where the industry spent a decade building SBOM standards, dependency scanning, and provenance verification — AI supply chain security is almost entirely unsolved.
OWASP classifies this as LLM03: Supply Chain — vulnerabilities in third-party components, pre-trained models, datasets, and AI integrations. It is one of the fastest-growing attack surfaces in enterprise security and one of the least defended.
Why This Matters Now
In 2024, researchers at Mithril Security published a demonstration they called PoisonGPT. Starting with Meta's open-source LLaMA model, they surgically modified a single layer of the model's weights to make it consistently return false information on a specific factual question — while passing every standard benchmark test without deviation. The modified model behaved normally on every other query. It would have passed any capability evaluation. The only way to detect the manipulation was to test the specific fact it had been poisoned to misrepresent.
They then uploaded the poisoned model to Hugging Face under a name nearly identical to a legitimate, trusted model.
PoisonGPT was a research demonstration. The attack it illustrated is not theoretical. Later that same year, security researchers scanning the Hugging Face model hub discovered more than 100 models containing embedded malware — backdoors, data exfiltration payloads, and arbitrary code execution capabilities hidden inside model files that appeared legitimate. These were not obscure repositories. Several had meaningful download counts.
Every enterprise that downloads a pre-trained model and deploys it without security scanning is relying on trust — in the platform, in the uploader, in the community review process — as its primary security control. That trust has already been violated.
What the CURVE™ Data Shows
The 2026 Stackcurve AI Security CURVE™ Report covers the AI Supply Chain Security category — vendors focused on securing open-source AI components, pre-trained models, datasets, and third-party integrations against poisoning, dependency confusion, malicious fine-tuning, and provenance attacks.
The market here draws heavily from the software supply chain security lineage — vendors including Snyk, GitGuardian, Scribe Security, and JFrog have extended their dependency and artifact security capabilities toward AI components. Purpose-built AI supply chain tools including ModelScan and the model security capabilities in the Palo Alto Protect AI platform provide scanning specifically for model files.
The full vendor rankings are in the 2026 AI Security CURVE™ Report — free to download.
What the CURVE™ data makes clear: coverage is improving but uneven. Scanning for malicious code embedded in model files is available today and underdeployed. Provenance verification — cryptographic attestation that a model file is what it claims to be, untampered, from who it claims to be from — is earlier and less consistent across the ecosystem. Dataset poisoning detection remains the least mature capability in the category.
The Gap Most Buyers Miss
Traditional software supply chain security is built on a principle: verify the provenance and integrity of every component before it enters your build pipeline. SBOMs (Software Bills of Materials), dependency scanning, code signing, and artifact registries implement this principle for software. The AI equivalent of each control exists in some form. Almost no enterprise applies them consistently to AI components.
The specific gaps Stackcurve sees most often:
Model files are not treated as executables. When an enterprise downloads a model file — a .safetensors or .gguf file, for example — they are downloading a data artifact that will run inference code within it. Model files can contain serialized executable code (particularly in older formats like Pickle-based .pt files). An organization that would never run an unsigned executable without a virus scan routinely deploys model files with no scanning whatsoever.
Fine-tuning pipelines are unsecured. When an enterprise fine-tunes a foundation model on proprietary data, the fine-tuning pipeline — the training scripts, the dataset processing code, the infrastructure — is a high-value target. Compromise of the fine-tuning pipeline can introduce subtle modifications to the resulting model without touching the foundation model itself. Most fine-tuning pipelines have no more security instrumentation than a developer's laptop.
Third-party AI integrations inherit implicit trust. LangChain plugins, LLM orchestration connectors, and AI-powered SaaS integrations often receive broad permissions — access to data, the ability to call APIs, connections to internal systems. The security review applied to these integrations is typically lighter than what would be applied to a traditional enterprise software integration, because they are perceived as "just AI tools."
Questions Your Buying Team Should Be Asking
1. Do you scan model files before deploying them to production? This is the minimum viable supply chain control for AI. Tools exist. The question is whether the process does. If the answer is no, this is the first thing to fix.
2. Where do your models come from, and how do you verify their provenance? "From Hugging Face" is not a provenance answer — it is a distribution channel. The question is: who originally trained the model, how was it transmitted to you, and can you verify it has not been modified in transit? Cryptographic model signing is the answer the industry is moving toward; ask your model providers where they are on that roadmap.
3. Have you applied an AI SBOM to your deployments? An AI SBOM — a bill of materials for the AI components in your stack — lists every model, dataset, framework, and plugin your AI system depends on, with version information and provenance. It is the foundation of supply chain visibility and the starting point for identifying compromised components if an incident occurs.
4. How is your fine-tuning pipeline secured? Apply the same access controls, audit logging, and integrity monitoring to your fine-tuning infrastructure that you apply to your production build pipeline. The output of that pipeline — your fine-tuned model — is as sensitive as the source code it was trained on.
5. What is your process for responding to a supply chain compromise in an AI component? If Hugging Face notified you tomorrow that a model you deployed last month had been found to contain malware, what would you do? If the answer is unclear, that incident response gap is worth closing before the notification arrives.
The Stackcurve Take
The PoisonGPT demonstration was alarming because it was so clean. No novel exploit. No sophisticated infrastructure. A publicly available model, a targeted weight modification, an upload to a public repository with a near-identical name to a trusted model. The entire attack required undergraduate-level ML knowledge and a Hugging Face account.
The defense is not sophisticated either: scan model files before you deploy them. Verify provenance where you can. Treat your fine-tuning pipeline as production infrastructure. Apply the same third-party risk process to AI integrations that you apply to other vendor software.
None of this is AI-specific insight — it is the software supply chain security playbook, applied to a new class of artifact that most security teams have not yet included in scope. The XZ Utils backdoor in 2024 demonstrated what a sophisticated supply chain attack on open-source infrastructure looks like. The AI supply chain is the same attack surface, with lower detection capability and faster adoption rates.
The question is not whether AI supply chain attacks will become a meaningful enterprise threat. They already are. The question is whether your security program has extended its scope to include the components your AI runs on.
The 2026 Stackcurve AI Security CURVE™ Report covers the AI Supply Chain Security category in detail. Download it free →
Stackcurve Advisory Briefs are independent research. No vendor pays for placement, tier assignment, or editorial influence. The CURVE™ methodology is disclosed in full at stackcurve.net/research/methodology.