137AI > Data Risks > AI Supply Chain Security

AI Supply Chain Security

The data risk category addressing the trustworthiness of the entire supply chain of parties, dependencies, build processes, and distribution infrastructure that produces and delivers AI updates. The category is the AI adaptation of software supply chain security, the discipline that became a major focus following incidents including the SolarWinds compromise and the Log4j vulnerability. The risk emerges because AI updates do not originate from a single trusted source; they emerge from deep chains of upstream dependencies, third-party components, build infrastructure, and distribution systems, any of which may be compromised in ways that affect the final update.

The category requires sharp distinction from related work covered separately. Model Update Integrity covers the integrity of the specific update artifact — whether this update is genuine, unmodified, and behaviorally sound. Data Transit Security covers update transit — whether the update is protected in motion. Supply-chain-of-updates covers the entire system that produces artifacts — every party, dependency, tool, build step, and distribution hop the update passed through. The distinction matters because a perfectly verified artifact can still be compromised if the supply chain that produced it was compromised; the compromise gets built into the artifact before the artifact is signed and verified.

What the AI Update Supply Chain Is

The AI update supply chain extends substantially beyond what conventional software supply chain analysis covers. Understanding the chain is foundational to understanding the risk.

The chain begins with upstream foundations. Base and foundation models, pre-trained components, training data sources, and broader upstream foundations form the starting point. Operators building on foundation models inherit the supply chain of those foundation models.

The chain includes the data dimension. Training datasets, fine-tuning datasets, evaluation datasets, retrieval corpora, and broader data inputs all form supply chain components with their own provenance considerations.

The chain includes the framework and tooling dimension. ML frameworks, libraries, training infrastructure, inference infrastructure, and broader software tooling form supply chain components that the conventional software supply chain analysis substantially covers.

The chain includes the model component dimension. Pre-trained components, adapters, LoRAs, third-party model modifications, embeddings models, and broader model components form AI-specific supply chain components.

The chain includes the agentic infrastructure dimension. Tools, plugins, MCP servers, and broader agentic infrastructure that AI agents depend on form supply chain components with specific considerations.

The chain includes the build dimension. Training infrastructure, fine-tuning infrastructure, build pipelines, and broader build infrastructure form the systems that produce updates.

The chain includes the distribution dimension. Model repositories, model hubs, vendor distribution infrastructure, package registries, and broader distribution systems form the systems that deliver updates.

The chain includes the compute dimension. Cloud infrastructure, specialized AI compute, and broader compute infrastructure form the systems that updates are produced and run on.

The aggregate AI update supply chain is substantially more complex than conventional software supply chains. The combination of conventional software supply chain components and AI-specific components produces a chain whose full extent operators may not have complete visibility into.

The AI-Specific Supply Chain Components

Several supply chain components are specifically AI-related and warrant treatment beyond conventional software supply chain analysis.

Component	Description	Supply Chain Considerations
Base and foundation models	The foundation models that operators build applications on through fine-tuning, prompting, or API consumption	Operators inherit the entire supply chain of the foundation model; limited visibility into foundation model training data and process
Training and fine-tuning data	Datasets used for training and fine-tuning including commercial datasets, open datasets, scraped data, synthetic data	Data poisoning attack surface; provenance often unclear for large datasets; licensing and consent considerations
Open-source model artifacts	Models, weights, and components distributed through model hubs including Hugging Face and equivalent repositories	Repository security; model authenticity; malicious model attack surface; serialization format vulnerabilities
ML frameworks and libraries	PyTorch, TensorFlow, JAX, Hugging Face Transformers, and broader ML software dependencies	Conventional software supply chain attack surface; dependency confusion; compromised package risk
Adapters and model modifications	LoRAs, adapters, fine-tuned variants, and third-party model modifications	Third-party modification trust; adapter authenticity; composition effects with base models
Agentic tools and MCP servers	Tools, plugins, and MCP servers that agentic AI systems depend on	Tool authenticity; MCP server security; action authority implications; emerging ecosystem with developing security practice
Build and training infrastructure	The infrastructure on which models are trained, fine-tuned, and built	Build infrastructure compromise produces compromised models; the SolarWinds-pattern risk applied to AI
Distribution infrastructure	Model repositories, hubs, registries, and vendor distribution systems	Distribution compromise produces compromised delivery; repository security; registry security
Compute infrastructure	Cloud and specialized compute infrastructure for training and inference	Infrastructure provider trust; hardware supply chain; specialized AI hardware considerations

The Dependency Depth Problem

AI systems have deep dependency chains that produce specific supply chain considerations. The dependency depth problem warrants direct treatment because it affects what supply chain visibility operators can realistically achieve.

Foundation models embody substantial inherited supply chain. An operator building on a foundation model inherits the foundation model's training data supply chain, the foundation model's framework dependencies, the foundation model's build infrastructure, and the broader foundation model supply chain. The inherited supply chain may extend substantially beyond what the operator has visibility into.

Each dependency has its own dependencies. ML frameworks depend on numerous libraries; those libraries depend on further libraries; the transitive dependency chain extends to substantial depth. Conventional software dependency analysis applies here with the additional AI-specific layers.

Data provenance is often unclear at depth. Large training datasets may aggregate data from numerous sources; the provenance of specific data may not be fully documented; the data supply chain may extend to depth that operators cannot fully trace.

Open-source model artifacts may have unclear provenance. Models distributed through hubs may be fine-tuned variants of other models, which are variants of other models; the model lineage may extend to depth that is not fully documented.

The visibility limit is structural. Complete supply chain visibility for AI systems is genuinely difficult; operators face supply chain depth that exceeds what current visibility infrastructure can fully address.

The trust delegation is structural. Operators that cannot achieve complete visibility delegate trust to upstream parties; the trust delegation is necessary but produces the supply chain risk that this category addresses.

The dependency depth problem affects what supply chain security can realistically accomplish. Comprehensive supply chain security addresses what is visible and manageable; the residual depth beyond visibility produces residual risk that operators navigate through trust delegation, vendor selection, and broader risk management.

Supply Chain Attack Categories

AI update supply chains face several specific attack categories that mitigation infrastructure must address.

Upstream model poisoning attacks compromise foundation models or pre-trained components before operators build on them. A poisoned foundation model produces compromised downstream applications regardless of downstream security practice; the attack operates at the most upstream point of the chain.

Training data poisoning in the supply chain compromises datasets that flow into training. The detailed treatment of training data poisoning appears in the broader data risks coverage. Supply chain data poisoning specifically addresses poisoning that enters through the data supply chain rather than through direct training data manipulation.

Compromised model repositories deliver malicious models to operators. Model hubs and repositories that are compromised may distribute malicious models; operators downloading models from compromised repositories receive the malicious models.

Malicious model uploads place adversarial models in legitimate repositories. Adversaries may upload malicious models to legitimate model hubs, potentially using names similar to legitimate models; operators may download the malicious models believing them legitimate.

Dependency confusion and typosquatting attacks exploit naming. Adversaries publishing malicious packages or models with names similar to legitimate ones may cause operators to install the malicious versions. The attack pattern is established in conventional software supply chains and applies to AI model and package ecosystems.

Compromised ML frameworks deliver malicious code through the framework dependency. Compromised versions of PyTorch, TensorFlow, or other frameworks would affect substantial portions of the AI ecosystem; the framework dependency is a high-value target.

Serialization format attacks exploit model file formats. Some model serialization formats including pickle-based formats can execute arbitrary code during deserialization; malicious models in vulnerable formats can compromise systems that load them. The safetensors format and similar approaches address this specific attack vector.

Build infrastructure compromise produces compromised models. The SolarWinds pattern — compromising build infrastructure to insert malicious code into legitimately-signed software — applies to AI. Compromised training or build infrastructure could produce compromised models that pass downstream verification.

Distribution infrastructure compromise affects update delivery. Compromised distribution infrastructure may deliver malicious updates, may deliver legitimate updates with modifications, or may suppress legitimate updates.

Malicious adapters and LoRAs compromise model behavior through model modifications. Third-party adapters and LoRAs that contain adversarial modifications affect models they are composed with.

Compromised MCP servers and tools affect agentic AI systems. MCP servers and agent tools that are compromised may produce adversarial effects on agents that use them; the agentic tool supply chain is an emerging attack surface.

The Open Model Supply Chain

The open model supply chain warrants direct treatment because open model artifacts are a substantial portion of contemporary AI supply chains and present specific considerations.

Model hubs including Hugging Face, and broader open model distribution infrastructure host substantial quantities of models, datasets, and components. The infrastructure has enabled substantial AI ecosystem development with corresponding supply chain implications.

Model authenticity on hubs is a specific consideration. Operators downloading models need to verify that models are what they purport to be; authenticity verification infrastructure on model hubs continues to develop.

Malicious models on hubs have been documented. Security research has documented malicious models uploaded to model hubs including models containing code execution payloads, models with backdoors, and broader malicious model patterns. Hub operators have implemented scanning and security infrastructure in response.

Serialization format security is a specific open model concern. The pickle format used by some model artifacts allows arbitrary code execution during loading; malicious models in pickle format can compromise systems that load them. The safetensors format developed substantially in response provides a safer serialization approach.

Model lineage and provenance on hubs is often incompletely documented. Models may be fine-tuned variants of other models; the lineage may not be fully documented; operators may not have complete visibility into model provenance.

Hub security infrastructure continues to develop. Hugging Face and other hub operators have implemented malware scanning, model security scanning, format security measures, and broader security infrastructure. The infrastructure continues to develop alongside the broader open model ecosystem.

The trust model for open models differs from vendor models. Open models distributed through hubs operate under different trust assumptions than vendor-hosted models; operators using open models take on supply chain verification responsibility that vendor relationships partially handle.

The open model supply chain produces both substantial value and substantial supply chain considerations. The value includes deployment flexibility, cost considerations, and broader benefits; the supply chain considerations include the verification responsibility operators take on.

Documented Incidents

Multiple documented incidents inform contemporary AI supply chain understanding.

The SolarWinds compromise discovered in 2020 represents the canonical software supply chain attack. Adversaries compromised SolarWinds build infrastructure to insert malicious code into legitimately-signed software updates distributed to substantial numbers of organizations including government agencies. The incident demonstrated that build infrastructure compromise produces compromised updates that pass downstream signature verification — the pattern that directly applies to AI build infrastructure.

The Log4j vulnerability disclosed in 2021 demonstrated the impact of vulnerabilities in widely-used dependencies. The vulnerability in the widely-used logging library affected substantial portions of the software ecosystem. The incident informed broader attention to dependency security including in AI contexts.

Malicious models on Hugging Face have been documented through security research. Specific research including work from JFrog, Protect AI, and others documented malicious models uploaded to Hugging Face including models with code execution payloads. The documentation informed hub security infrastructure development.

PyTorch dependency compromise occurred in December 2022 when a malicious package using dependency confusion affected the PyTorch nightly build supply chain. The incident demonstrated AI framework supply chain vulnerability.

Pickle format exploitation has been substantively documented. Security research has demonstrated arbitrary code execution through malicious model files in pickle format; the research informed the development and adoption of the safetensors format.

The xz Utils backdoor discovered in 2024 demonstrated a sophisticated multi-year supply chain compromise of a widely-used compression library. The incident demonstrated the patience and sophistication of supply chain attackers and informed broader supply chain security attention.

Npm and PyPI malicious package incidents have been recurring across the conventional software supply chain with implications for AI software dependencies. Specific incidents of malicious packages, dependency confusion attacks, and broader package supply chain attacks inform AI dependency security practice.

Model hub typosquatting incidents have been documented where adversaries upload models with names similar to legitimate popular models. The incidents inform operator practice on model source verification.

The aggregate documented landscape continues to develop. Both conventional software supply chain incidents with AI implications and AI-specific supply chain incidents inform ongoing operator practice.

Supply Chain Security Frameworks

Multiple frameworks provide structured methodology for supply chain security with application to AI updates.

SLSA (Supply-chain Levels for Software Artifacts) provides a framework for build and distribution integrity through progressive levels of supply chain security maturity. The framework addresses build integrity, provenance, and broader supply chain security with application to AI artifact supply chains.

SBOM (Software Bill of Materials) provides structured inventory of software components. The SBOM approach supports understanding of what components a software artifact includes. AI-BOM (AI Bill of Materials) and model bill of materials concepts extend the SBOM approach to AI-specific components including models, datasets, and broader AI components.

NIST Secure Software Development Framework (SSDF, NIST SP 800-218) provides framework for secure software development including supply chain security practices. NIST has been developing AI-specific extensions including SSDF practices for generative AI.

NIST Cyber Supply Chain Risk Management (C-SCRM, NIST SP 800-161) provides framework specifically for supply chain risk management. The framework supports structured supply chain risk practice with application to AI supply chains.

Executive Order 14028 (2021) on improving national cybersecurity included substantial software supply chain provisions including SBOM requirements for federal software procurement. The framework has shaped broader supply chain security practice including for AI.

The Secure Software Development Attestation framework requires software producers to attest to secure development practices for federal procurement. The framework includes provisions relevant to AI software supply chains.

Sigstore provides signing and verification infrastructure with transparency log infrastructure. The infrastructure supports artifact signing and verification with application to AI artifact supply chains.

The CISA supply chain security guidance and broader CISA infrastructure provides supply chain security guidance with application to AI.

EU Cyber Resilience Act includes supply chain security provisions for products with digital elements including AI products placed on the EU market.

The Coalition for Secure AI (CoSAI) and broader industry initiatives have been developing AI-specific supply chain security practice. The initiatives address AI supply chain security with industry coordination.

The aggregate framework landscape continues to develop with substantial AI-specific extension activity. Conventional supply chain security frameworks provide foundation; AI-specific extension addresses the AI-specific components.

Provenance and Attestation Infrastructure

Provenance and attestation infrastructure supports supply chain security by establishing verifiable claims about how artifacts were produced.

Build provenance establishes verifiable claims about how an artifact was built. Provenance attestation including SLSA provenance, in-toto attestation, and broader provenance infrastructure supports verification that an artifact was built through the expected process on the expected infrastructure.

Model provenance extends build provenance to AI-specific dimensions including training data provenance, training process provenance, and broader model production provenance. Model provenance infrastructure continues to develop.

Model cards and documentation provide structured provenance information. The detailed treatment appears in AI Documentation as Compliance Evidence. Model cards support both human and increasingly automated provenance verification.

Artifact signing establishes authenticity. Signed artifacts can be verified as genuinely from the legitimate producer; signing infrastructure including Sigstore, code signing approaches, and model-specific signing supports authenticity verification.

Transparency logs provide verifiable public records of artifact production. Transparency log infrastructure including Sigstore's Rekor supports verification that specific artifacts were recorded in public logs.

Dependency tracking through SBOM and AI-BOM supports understanding of what components artifacts depend on. Comprehensive dependency tracking supports both vulnerability management and supply chain risk management.

Data provenance infrastructure addresses the data dimension specifically. Establishing verifiable provenance for training data supports the data supply chain security dimension; the infrastructure continues to develop given the difficulty of data provenance for large datasets.

The aggregate provenance and attestation infrastructure supports supply chain security. The infrastructure cannot establish complete supply chain trust but substantially supports supply chain risk management.

The Vendor Concentration Dimension

The AI supply chain has substantial concentration in a limited number of foundation model providers. The concentration dimension warrants direct treatment because it affects the systemic supply chain risk profile.

Foundation model concentration means substantial portions of AI deployment depend on a limited number of foundation model providers. Operators across many applications build on the same foundation models; the concentration produces correlated supply chain risk.

The correlated risk dimension is structurally significant. A supply chain compromise affecting a major foundation model provider would affect substantial portions of the AI ecosystem simultaneously; the correlated exposure exceeds what diversified supply chains would produce.

Compute concentration adds another concentration dimension. AI compute is concentrated in a limited number of cloud providers and specialized compute providers; the compute concentration produces additional correlated supply chain risk.

The hardware concentration dimension affects the broader supply chain. AI accelerator hardware is concentrated in a limited number of providers; the hardware supply chain concentration produces additional considerations.

The framework concentration dimension affects software supply chain. The ML framework landscape is concentrated in a limited number of major frameworks; framework concentration produces correlated software supply chain risk.

The model hub concentration affects open model supply chain. Open model distribution is substantially concentrated in a limited number of major hubs; hub concentration produces correlated open model supply chain considerations.

The concentration produces both efficiency and systemic risk. The concentration enables substantial ecosystem efficiency; the concentration also produces systemic supply chain risk that diversified supply chains would not produce.

The concentration dimension affects what supply chain risk management can accomplish. Individual operator supply chain practice cannot address the systemic concentration; the systemic dimension requires ecosystem-level and policy-level engagement beyond individual operator practice.

What Supply Chain Security Cannot Prevent

Supply chain security has substantial limits that operators should engage directly.

Sophisticated upstream compromise may not be detectable. The SolarWinds and xz Utils incidents demonstrated that sophisticated, patient adversaries can compromise supply chains in ways that evade detection for substantial periods; some upstream compromise may not be detectable through available infrastructure.

Dependency depth beyond visibility produces residual risk. Operators cannot achieve complete supply chain visibility; the residual depth beyond visibility produces residual risk that supply chain security cannot eliminate.

Foundation model supply chain may be substantially opaque to operators. Operators building on foundation models have limited visibility into the foundation model supply chain; the opacity produces residual risk regardless of downstream supply chain practice.

Data provenance may be genuinely unverifiable for large datasets. Establishing complete provenance for large training datasets is genuinely difficult; the data supply chain may include components whose provenance cannot be verified.

Zero-day vulnerabilities in dependencies cannot be prevented through supply chain security. Supply chain security addresses known issues and verification; novel vulnerabilities in dependencies produce risk that supply chain security does not prevent.

Concentration risk cannot be addressed through individual operator practice. The systemic concentration dimension exceeds what individual operators can address; the systemic risk requires ecosystem and policy engagement.

Insider threats within supply chain parties produce risk that external verification may not detect. Compromise originating from insiders at supply chain parties may not be detectable through the verification infrastructure operators deploy.

The aggregate supply chain security limits produce specific implications. Mature operators combine supply chain security infrastructure with broader risk management including vendor selection, defense in depth, monitoring, and incident response rather than relying on supply chain security alone.

Specific Concerns for Operators

Operators managing AI update supply chains face several recurring considerations.

Supply chain inventory addresses what the operator's AI supply chain actually includes. Operators benefit from explicit inventory of supply chain components including foundation models, data sources, frameworks, model artifacts, tools, and broader components.

SBOM and AI-BOM practice supports dependency understanding. Maintaining bill of materials for AI systems supports both vulnerability management and supply chain risk management.

Source verification addresses where supply chain components come from. Verifying model sources, verifying package sources, and broader source verification supports supply chain security.

Artifact verification addresses whether artifacts are authentic. Signature verification, hash verification, and broader artifact verification supports detection of tampered or malicious artifacts.

Serialization format practice addresses the specific format security dimension. Preferring safer serialization formats including safetensors over vulnerable formats reduces specific attack surface.

Vendor evaluation includes supply chain practice. Operators evaluating AI vendors may consider vendor supply chain security practice, vendor provenance infrastructure, and broader vendor supply chain considerations.

Dependency management addresses the conventional software supply chain dimension. Dependency scanning, vulnerability management, and broader dependency management practice applies to AI software dependencies.

Provenance practice addresses establishing and verifying provenance. Both producing provenance for operator-built artifacts and verifying provenance for consumed artifacts supports supply chain security.

Monitoring addresses detection of supply chain issues. Monitoring for supply chain anomalies, dependency vulnerabilities, and broader supply chain indicators supports response.

Incident response preparation addresses supply chain incidents specifically. Supply chain incidents may require response that differs from conventional incident response; pre-prepared infrastructure supports effective response.

The Reframe

Supply-chain-of-updates addresses the trustworthiness of the entire system that produces AI updates — every party, dependency, build step, and distribution hop — as distinct from the integrity of the final artifact or the security of its transit. The SolarWinds pattern applies directly: a compromised build or upstream component produces a compromised model that passes downstream signature verification, because the compromise was built in before signing. The dependency depth and vendor concentration dimensions produce residual and systemic risk that individual operator practice cannot fully address.

Related Coverage

Data Risks | Model Update Integrity | Data Transit Security | Cybersecurity