AI model supply chain security review

Why buyers commission an AI model supply chain security review

A customer questionnaire, an insurer renewal, a launch date or a due diligence request has started asking how third-party models and datasets are controlled. An AI model supply chain security review answers that question with evidence. It is a review engagement that examines where models, adapters, datasets and AI libraries come from, how they are verified before use, where they are stored, and how they are updated and retired. Findings are mapped to LLM04 Supply Chain and LLM05 Data and Model Poisoning in the OWASP Top 10 for LLM Applications 2026. It is not a live attack test: the work is document review, interviews and configuration inspection, with the team checking claims against what the pipeline actually does.

The risks sit in places that most organizations do not inventory. A pretrained model pulled from a public hub may come with serialized code that runs on load, a fine-tuned adapter may have been trained by a contractor on data nobody has traced, and a training corpus may have been scraped, merged or relabeled without a record of who changed it. An AI library pinned loosely in a build file can change under the team without notice. A review asks which of these artifacts are used, who approved them, whether checksums or signatures are verified against a trusted source, and whether the team could rebuild a model from known inputs if one were found to be compromised.

Before commissioning this work, ask the provider to confirm which artifacts are in scope, including the model registry, the fine-tuning pipeline, the data labeling process and the package sources used by the serving stack. Ask how the review will treat models your own team trained versus those you downloaded. Confirm whether the output will be a written gap assessment with prioritized remediation, and whether your auditor or customer expects a specific framework reference. The cost and duration depend on how many artifacts, pipelines and teams are involved, so a fixed scope agreed in writing before work starts is the basis for a fair quote.

What can be in scope for an AI supply chain review

A supply chain review is only as useful as its boundary. The decisions below set which systems, artifacts, pipelines and people the reviewer examines, and they should be written down before any work starts. Each one changes the evidence the reviewer needs and the effort the review takes.

AI systems and their owners

Start by naming the AI systems whose models, adapters or datasets are under review, and the business owner of each. Include internal models, vendor-hosted models called through an API and any system that pulls weights or data from a public hub. Ask which systems are high-risk under your own policy, since that often decides depth. Confirm whether shadow deployments by product teams are in or out of the boundary.

Model and dataset artifacts

List every artifact type the systems depend on: base models, fine-tuned models, LoRA or other adapters, training and evaluation datasets, and AI libraries and runtimes. Each type has different provenance questions. A dataset may have no checksum at all, while a library has a version history and a dependency tree. Record which artifacts are first-party, which come from third parties and which were generated synthetically.

Pipelines that move artifacts

Map the path each artifact takes: download from an external source, conversion between formats, fine-tuning or evaluation runs, and promotion to production. Each step can change the artifact or its trust status. Conversion and fine-tuning are common points where code runs on loading and where training data changes behavior. Ask whether the reviewer can trace one artifact from source to deployment.

Storage locations and registries

Define where artifacts are kept: model registries, object stores, notebook volumes and container images that bundle weights. Check who can write to each location, whether writes are logged and whether an artifact can be replaced after approval without a new review. Include backups and developer caches, which often hold unreviewed copies. Confirm which of these run inside your cloud account and which sit with a hosting vendor.

People with publish and approve rights

Identify everyone who can publish, sign, approve or promote an artifact, including service accounts and CI runners. Separation of duties is the main question: the person who fine-tunes a model should not be the only one who approves its release. Record how access is granted and removed, and how approvals are evidenced. This list is often longer than the team expects once automation is counted.

Rules of engagement and access

Decide what the reviewer may touch. Options include reading artifact stores and pipeline configuration directly, or receiving only documentation, screenshots and exports that you prepare. Direct access gives stronger evidence but needs access controls, data handling terms and sometimes restrictions on copying proprietary weights or training data. Confirm the terms in writing, including which vendor-hosted services fall inside your own contracts. The agreed scope is recorded in writing before work begins.

How the review runs from scoping to retest

The review moves from agreed scope and a complete artifact inventory through provenance, integrity, storage and dataset checks to a prioritized report and a retest of agreed fixes. Each phase produces a record the client can use to decide what to change first.

Agree scope and inventory every artifact
DeployOpen and the client agree in writing which models, adapters, datasets, embedding models, container images and AI libraries fall inside the review, along with the systems that build, store and deploy them. The client's engineers and data owners then produce an inventory listing each artifact, its version, its source and the team responsible for it. The step is complete when the scope document and the inventory are signed off by both parties. Ask the provider how timing is set out in the written scope, since this review does not run to a fixed duration in every case.
Trace sourcing and provenance records
Reviewers examine where each model, adapter, dataset and library came from, including upstream repositories, model cards, licenses, publisher identity and any intermediate hosts or mirrors. They check whether the client can show the lineage of each artifact from origin to the deployed copy, and whether third-party components are pinned to specific versions. Relevant to LLM04 Supply Chain, this step also covers the agentic dimension in ASI04 Agentic Supply Chain Vulnerabilities, such as plugins, tool servers and agent templates pulled in at runtime. The output is a provenance map that marks each artifact as traceable, partially traceable or untraceable.
Examine verification and integrity controls
Reviewers assess how the client confirms that an artifact is the one the publisher released. This includes checksums, cryptographic signatures, attestation of build pipelines, pinned hashes in configuration, and whether verification runs automatically before a model is loaded. They also look at how unsafe serialization formats are handled and whether loaders execute code from model files. The step closes with a list of control gaps, each tied to the artifact it affects and the evidence that shows the gap exists, such as a configuration file or pipeline log.
Review storage, access and update processes
Reviewers look at where artifacts are stored, who can write to those locations, how changes are approved, and how upgrades and rollbacks are made. They check registry and bucket permissions, separation between development and production stores, audit logging for artifact changes, and whether automated update tools can pull new versions without review. Where a self-hosted model registry is used, the review covers its authentication and its own patch cycle. The output is a map of who can change each artifact and what record is left behind when they do.
Assess fine-tuning and dataset handling against LLM05
This step covers LLM05 Data and Model Poisoning, which concerns risks from training data, fine-tuning sets and pipelines that alter model behavior. Reviewers examine how datasets are collected, filtered and labeled, who may contribute data, whether contributions are validated and attributed, and how fine-tuning jobs are reproduced. They also check whether the client keeps evidence of the data used for each model version. The review does not run poisoning attacks. It finishes with a list of places where untrusted data can reach training or adaptation and the controls that currently limit that path.
Report findings and remediation priorities
DeployOpen issues a written report that states each finding, the artifact or process it applies to, the evidence reviewed, the OWASP LLM Top 10 2026 category it maps to, and a suggested remediation. Findings are ranked by the risk they carry to the deployed system and by how difficult each fix is, so the client can plan sequencing. The report also records what was reviewed and what was out of scope. It is complete when the client has accepted the findings in a readout meeting and the priorities are logged. Ask the provider whether the report format suits the client's internal risk register or auditor requests.
Retest agreed fixes and close findings
Once the client has implemented fixes, DeployOpen checks each one against the finding it addressed, using the same evidence method as the original review. A finding closes when the retest shows the control is in place and working as described. Findings that remain open are recorded with the reason, and any accepted risks are documented with an owner. The closing record lists each finding, its status and the evidence used, which the client can supply to auditors or customers. Retest scope and any additional cost should be stated in the written agreement before the retest begins.

What weaknesses look like in each artifact type

Model weights are the largest and most opaque artifact in an AI system. Most organizations take a base model from a public hub or a commercial provider, sometimes fine-tune it internally, and sometimes convert it to another format for serving. Each step is a point where the origin of the file, its integrity and its training lineage can be lost. Under LLM04 Supply Chain, the review asks whether the organization knows exactly which weights are running in production and how they got there. Under LLM05 Data and Model Poisoning, it asks whether the training and fine-tuning history can be traced and whether anyone checks behavior after changes.

Unverified downloads: weights pulled from a hub without checksum comparison, publisher verification or a pinned revision.
Conversion steps: format conversions and quantization performed by people or scripts with no record of the tool version, input hash and output hash.
Unclear fine-tuning lineage: fine-tuned models without a record of the base revision, training data version and the person or pipeline that approved the run.
Unsigned internal artifacts: model files copied between buckets or registries without signatures or access logs.
Missing behavioral baseline: no evaluation results stored for each version, so a silent change goes unnoticed.

Evidence to request: a model inventory with source, revision and hash for each deployed model, conversion and fine-tuning logs, and the evaluation results that were run on each version. Confirm with the team which registry is the system of record.

What drives the cost and duration of this review

This table compares the factors that change the effort of a model and dataset supply chain review, why each one matters, and how to keep scope efficient. Read it before requesting a quote, because most of the cost sits in artifact counts, verification evidence and how many parties touch the pipeline. DeployOpen does not publish a price list; a quote follows from the written scope.

Driver	Why it changes effort	How to scope efficiently
Number of AI systems and model artifacts	Each model, adapter and fine-tuned checkpoint needs its provenance traced, its file format and loading path reviewed, and its hashes or signatures checked. Effort grows with each artifact in the inventory.	Agree a named list of production and near-production artifacts in writing. Exclude experimental checkpoints or state how they are sampled.
Number of dataset sources and labeling vendors	Each source needs its licence, collection method, consent basis and change history examined. Each labeling vendor adds a contract, access model and quality control to review, which is where poisoning risk is most often hidden.	Group sources by owner and ingestion route. Ask for a data flow diagram before the review starts so vendors are listed once.
Number of pipelines and model registries	Every build, fine-tuning job, conversion step and registry is a place where artifacts can be replaced or altered. Access controls, promotion rules and logging must be checked at each point.	Limit the review to pipelines that feed production. Ask the client to confirm which registries are authoritative and which are mirrors.
Fine-tuning done in-house or by third parties	In-house fine-tuning means the client's own training data, compute and code are reviewed. Third-party fine-tuning adds contractual terms, hand-off evidence and data-handling obligations that must be verified rather than assumed.	Ask whether the third party supplies build logs, dataset manifests and signed outputs. Where they do not, state that gap as a finding and scope accordingly.
Depth of verification evidence available	Where hashes, signatures, model cards, dataset datasheets and provenance records already exist, review time falls. Where they are missing, the reviewer must reconstruct history from logs and interviews, which takes longer.	Request the evidence index before the quote. Ask for existing records to be shared in advance so interview time is spent on gaps.
Interviews with model owners and data engineers	Provenance questions often need the people who selected, cleaned or retrained an artifact. Each interview round adds scheduling and note-taking effort on both sides.	Name interviewees and their areas in the scope. Ask for one consolidated question list per team rather than ad hoc sessions.
Retest rounds after remediation	Each remediation cycle requires re-checking changed controls, updated artifacts and new evidence. A fixed retest allowance is usually cheaper than open-ended re-review.	Set the number of retest rounds and what each round covers in the written scope, along with the evidence the client must supply before each one.
One-time versus recurring effort	The initial review and report carry the largest share of effort, because inventory, provenance tracing and control mapping happen once. Periodic re-review recurs as models are retrained, datasets change or new vendors are added, and is usually smaller in scope.	Ask for the initial review and each recurring re-review to be quoted as separate phases with separate assumptions.

Which frameworks expect supply chain and poisoning controls

This table compares the frameworks and regulations that address model and dataset supply chain risk and data or model poisoning, and what each one expects you to show. Read each row as a question to ask your assessor or customer: which obligation applies to you, and what evidence will they ask for. The main takeaway is that a supply chain review prepares evidence for these expectations, while independent auditors and certification bodies decide conformance.

Framework or program	What it expects on supply chain and poisoning	Evidence to prepare
EU AI Act Article 15(5)	High-risk AI systems must be resilient against attempts to exploit vulnerabilities, including data poisoning, model poisoning, adversarial examples, model evasion and confidentiality attacks.	Inventory of training, fine-tuning and validation datasets with provenance, poisoning-focused test results and documented resilience measures per system.
EU AI Act Article 55(1)	Providers of general-purpose AI models with systemic risk must perform and document adversarial testing. Confirm with counsel whether your model meets this threshold.	Written adversarial testing plan, test records, findings and remediation log, and a record of third-party or upstream model components tested.
OWASP LLM04 Supply Chain (2026)	Covers risk from third-party models, adapters, libraries, plugins and hosting services, which can introduce vulnerable or tampered components.	Component inventory (models, adapters, libraries), source and license records, integrity checks, and a documented process for approving and updating each component.
OWASP LLM05 Data and Model Poisoning (2026)	Covers corrupted training, fine-tuning, embedding or retrieval data and pipelines that can change model behavior or outputs.	Dataset lineage records, access controls on data pipelines, data validation and anomaly checks, and evidence of behavioral testing after each training change.
NIST AI 100-2 E2025	Describes poisoning attacks on predictive and generative AI, including large language models, and the mitigations that address them. Use it as a shared taxonomy for threat scenarios.	Threat model mapping each poisoning category to your system, with the mitigations chosen, the reason for each choice and the residual risk accepted.
NIST AI 600-1 (Generative AI Profile)	Voluntary guidance for incorporating trustworthiness into the design, development, use and evaluation of generative AI, including third-party and content risks.	Governance records showing who approves third-party models and data, documented risk decisions, and evaluation results tied to each release.
HITRUST AI security work	HITRUST has an AI Security Certification and a recent Agentic AI factor within its CSF. Confirm current scope and applicability directly with HITRUST before relying on it.	Control mapping prepared for the assessor, showing which supply chain and poisoning controls exist and where they are documented. A review does not issue a HITRUST certification.

What the client receives and how it reaches an auditor or customer

A supply chain review produces a small set of connected records, each built on the one before it. The inventory defines what was examined, findings explain what was wrong, and the remediation and retest show what changed. DeployOpen prepares this evidence, but it does not issue audit reports or attestations, so confirm with your auditor or customer which format they accept before scope is signed.

Inventory of in-scope artifacts

A register of every model, adapter, dataset, embedding model and AI library in scope, with its source, version or revision identifier, license, and how it entered your environment. For each item the inventory records who approved its use and where it is stored. Check that the list matches what your build and deployment pipelines actually pull, not only what the architecture diagram shows. Gaps here make every later finding harder to trust.

Findings with severity and evidence

Each finding names the weakness category, the affected artifact or pipeline stage, and the evidence that supports it, such as configuration exports, pipeline logs, or repository settings. Severity ratings follow a scale you agree in writing before testing begins. Ask for each finding to state whether it was confirmed in your environment or inferred from documentation, because auditors and security teams treat those differently.

Remediation plan with owners

A prioritized plan that assigns each finding to a named owner, such as the data science lead, platform team or procurement function, with a recommended order of work. It separates quick configuration changes from changes that need new approval steps or re-sourcing a model. Ask that the plan distinguish one-time fixes from recurring controls, such as re-verifying model hashes after each upstream update.

Retest results for agreed fixes

A targeted retest confirms whether the fixes you agreed to make actually close the findings. Each retested item receives a result, such as closed, partially closed or open, with the evidence used to reach that result. Retesting covers only the findings and fixes named in the agreed scope, so ask in advance whether a retest is included in the fixed price or billed separately.

Statement of scope, assumptions and limitations

A written statement that sets out what was reviewed, the time window, the systems and sources included, and what was excluded. It also records assumptions, such as access to vendor documentation or to a model registry, and the limitations that follow from them. Auditors use this section to judge how far the findings can be relied upon, so it should be read as carefully as the findings themselves.

Evidence pack for auditors and customers

A compiled package that brings the inventory, findings, remediation status and retest results together with an index, so an auditor or customer security team can trace each claim to its evidence. It is prepared by DeployOpen and is not an audit report, attestation or certification. Confirm with your assessor whether they want this pack, a mapping to specific control identifiers, or a different format before the review starts.

Questions buyers ask before commissioning an AI supply chain review

What does the review need from our team to get started?

The review needs named owners and documentation more than it needs engineering time. Expect to provide an inventory of every model, adapter, dataset and AI library in use, the source each one came from, and the pipeline that fine-tunes or deploys them. Useful inputs include:

Model cards, licence terms and download records for third-party models
Dataset provenance, labelling process and data handling records
Build and deployment pipeline configuration, plus a list of who can approve changes

Agree on these owners and artifact lists in the written scope before work begins.

How long does an AI supply chain review take, and what drives the timing?

Timing depends on how many models and datasets are in scope, how many pipelines build or update them, and how quickly your team can produce the documentation. Fixed-scope engagements agree the timeline in writing once the inventory is known, so ask for the schedule to be tied to specific deliverables. Timing usually slips when provenance records are missing, because reconstructing where an artifact came from takes longer than reviewing it. Ask the provider to state which inputs the schedule assumes.

Can our team run this review internally using OWASP and NIST guidance?

Yes, many teams start internally, and the guidance is public. The OWASP Top 10 for LLM Applications 2026 covers supply chain risk at LLM04 and data and model poisoning at LLM05, and NIST AI 100-2 E2025 describes poisoning attacks and mitigations. An internal review is worth doing if you have people who understand both your pipelines and these risks. An external review adds an independent view of verification gaps that teams often miss in their own work, and it produces a report that customers and assessors may accept more readily.

Do we need a live attack test as part of this review?

No. A supply chain review examines documents, configurations, pipelines and controls, and it does not attack your running systems. If you also want to know whether an adversary could exploit a weakness, that is a separate engagement. Prompt injection testing covers direct and indirect injection against an application, and AI red teaming tests against agreed harm scenarios using MITRE ATLAS techniques. Ask the provider to state in the scope which activities are excluded.

How often should we repeat the review as models and datasets change?

Repeat the review whenever a material change occurs, such as a new base model, a new fine-tuning dataset, a change of model source or a new library in the training or serving path. Many organizations also review on a fixed schedule set by their risk policy or compliance framework. Recurring work is usually a retainer, where the provider re-checks changed artifacts rather than repeating the full inventory. Confirm with the provider how changes are detected and what a periodic review costs in effort.

What does a retest of the review cover?

A retest checks whether the agreed remediations were implemented and whether they work as described. It does not repeat the full review unless the scope says so. Typical retest items include updated verification steps for model and dataset downloads, changes to access control on model registries, and documented ownership for each artifact. Ask the provider to confirm in writing which findings are covered by the retest, how evidence will be supplied, and whether a retest fee or extra days are included in the quote.

What does our auditor or customer receive at the end of the review?

You receive a written report that lists the artifacts reviewed, the findings, the evidence examined and the recommended remediations. DeployOpen does not issue audit reports or certifications. Independent auditors and certification bodies do, so the report supports your assessment rather than replacing it. Before work starts, ask how the report maps to the control areas your assessor or customer uses.

What your team should prepare before the review starts

The most useful preparation is a written inventory of every model, adapter, dataset and AI library in use, including those embedded in vendor products and those pulled in at build time. For each item, record where it came from, the version or revision in use, the license, and the person who approved it. Where a model or dataset was fine-tuned, record the base artifact, the training or tuning data sources, and the pipeline that produced the result. Alongside the inventory, gather the pipeline and registry documentation: which model hub, artifact store or package index is used, how artifacts move between environments, and any mirrors or caches. Existing hash or signature records are valuable even if incomplete, because the reviewer can see which artifacts can already be checked against a known digest and which cannot.

Access records are the second input. Ask for a current list of who can publish, promote or approve artifacts into each environment, along with any service accounts and CI/CD identities with the same rights, and the date each grant was last reviewed. Name a technical owner for each AI system in scope, someone who can explain its data flows and update decisions without a meeting. Also name a single point of contact who will answer questions during the engagement and has authority to approve evidence requests, since delays in this channel are a common cause of slipped timelines.

Confirm in the written scope whether any change freezes or model release windows apply during the review, and whether the reviewer may read pipeline configuration, registry settings and access logs directly or only through exports. Agree which environments are covered, and whether a snapshot of the inventory is frozen at kickoff. Ask DeployOpen to state in the scope exactly which evidence you must supply, in what format and by which date, so that owners can plan their time before work begins.

What to check when choosing a provider for this review

A supply chain review touches model weights, training data and the pipelines that move them, so the questions you ask a provider matter as much as the answers. The cards below cover experience, framework fluency, the line between review and testing, handling of sensitive artifacts, written scope and ongoing re-review.

Evidence of pipeline review experience

Ask for a written description of the method the provider uses to trace where models, adapters, datasets and AI libraries come from, plus a redacted sample report structure. Ask how it records provenance, verification and update decisions for each artifact. Do not accept invented case studies or vague claims of experience. Ask which parts of the method the provider has applied to model supply chains specifically, not only to general software dependencies.

Fluency in OWASP, ATLAS and NIST

Ask the provider to map its findings to LLM04 Supply Chain and LLM05 Data and Model Poisoning in the OWASP LLM Top 10 2026, to relevant MITRE ATLAS techniques, and to the risk categories in NIST AI 100-2. A credible provider can explain how each framework divides the same risk and where they overlap. Vague references to frameworks without a mapping in the deliverable are a warning sign.

Reviewed versus tested boundaries

A supply chain review examines documents, configurations, pipelines and controls. It does not attack a live model. Ask the provider to state in writing what it reviews, what it samples, what it tests and what it will not examine. Confirm whether any hands-on validation, such as checking a model hash against a published source, is included and what it covers.

Handling of weights and training data

Model weights and training datasets can be proprietary, licensed or personal data. Ask how the provider requests access, where artifacts are stored during review, who can view them, how long they are retained and how they are destroyed. Confirm that a confidentiality agreement is signed before any artifact is shared, and that the review can proceed using metadata and hashes where full access is not needed.

Fixed scope and deliverables in writing

Ask for a written scope that lists the artifacts in scope, the pipelines and repositories covered, the frameworks used, the deliverables and the acceptance criteria. Check that any changes to scope are handled through a written change request. Ask what the provider will not cover, such as the upstream training process of a third-party model, so the gaps are visible before work starts.

Retest and periodic re-review

Models, adapters and libraries change often, so a one-time review can go stale quickly. Ask whether the provider includes a retest of findings, whether it offers periodic re-review when a model version or dependency changes, and how it is priced and scheduled. Also ask the provider which of its claims it cannot support, and confirm any vendor partnerships directly with the vendor rather than relying on the provider's word.

How DeployOpen scopes, reviews, reports and retests AI supply chains

DeployOpen treats AI model and dataset supply chain review as a fixed-scope review engagement, not a live attack test. We agree in writing which models, adapters, datasets and AI libraries are in scope, then examine how they are sourced, verified, stored and updated against LLM04 Supply Chain and LLM05 Data and Model Poisoning. Findings are documented so your team and your assessors can follow the evidence.

How we scope

The written scope lists each model, adapter, dataset and AI library in the review, with its origin and the systems that consume it.
We agree which pipelines are in view, such as fine-tuning, data ingestion and model promotion, before any review work starts.
Engagements are fixed-scope, with deliverables and timelines confirmed in writing; ongoing support is available as a retainer.

How we build

We review how artifacts are obtained, including source registries, licensing terms and whether provenance records are kept for each item.
We examine verification controls such as checksums, signatures, and the handling of pickled or serialized model files in your environment.
We assess storage, access control and update procedures, and how changes to models and datasets are approved and traced.

How we hand over

You receive a written report mapping each finding to LLM04 or LLM05, with the evidence, affected components and the reasoning behind each rating.
Recommendations are prioritized so your engineering team can assign owners and decide what to change first.
Documentation of the reviewed supply chain is delivered in a form you can share with auditors or customers who ask how AI artifacts are controlled.

How we support

After remediation, we can retest the specific items and controls named in the original report, and document what changed.
Continuous review of new models, datasets and library updates is available under a retainer agreed in writing.
Confirm with us early whether your auditor or customer requires a particular format, so the handover matches what they will request.