AI Model and Artifact Integrity - Phase 2: Build and Deploy
AI-enabled applications increasingly incorporate pretrained models, fine-tuned models, embeddings, model adapters, and supporting software artifacts obtained from external repositories.
These artifacts introduce supply chain security risks, including compromised models, unauthorized modifications, unsafe serialization formats, and untrusted dependencies.
During Build and Deploy, organizations should verify that AI models and supporting artifacts originate from approved sources and have not been modified unexpectedly.
This activity extends established software supply chain security practices to AI models, associated dependencies, and release artifacts.
AI Model and Artifact Integrity
Objective: Ensure AI models and supporting software artifacts are verified for provenance, integrity, authenticity, and known security risks before packaging or deployment.
The NIST Secure Software Development Framework (SSDF) establishes practices for protecting software artifacts and verifying software integrity.
The NIST AI SSDF Profile (SP 800-218A) addresses additional security considerations associated with AI model development and integration.
The OpenSSF Model Signing project provides open-source capabilities for signing and verifying AI model artifacts.
For DevOps teams, model verification should become an automated CI/CD activity similar to container signature verification, dependency scanning, and software artifact validation.
Model signatures and checksums help establish artifact integrity, but do not independently prove that a model is free of malicious behavior or vulnerabilities. Additional security assessments remain necessary.
AI Security Requirements and Open-Source Tooling
| Framework Reference | Security Activity | CI/CD Implementation | Suggested Open-Source Tools |
|---|---|---|---|
| NIST SSDF PS.2; OWASP LLM03 | Model provenance verification | Verify model origin, approved distribution source, version, and associated provenance records before use. | OpenSSF Model Signing, in-toto |
| NIST SSDF PS.2 | Cryptographic model integrity | Validate cryptographic hashes or signatures to detect unauthorized modifications to model artifacts. | OpenSSF Model Signing, Sigstore Cosign |
| NIST SSDF PW.4; OWASP LLM03 | Model file security | Scan supported AI model formats for potentially malicious serialized content and unsafe loading mechanisms. | ModelScan |
| NIST SSDF PW.4, PW.7 | Model dependency vulnerability analysis | Scan frameworks, libraries, containers, and supporting software components for known vulnerabilities. | Trivy, OSV-Scanner |
| NIST SSDF PS.3 | Model and dependency inventory | Record model identifiers, versions, approved sources, and supporting software dependencies. | Syft, CycloneDX |
| NIST SSDF PS.2, PS.3 | Model version traceability | Associate model versions and approved artifacts with application release records. | DVC, Git LFS |
| NIST SSDF PS.2 | Build provenance and attestations | Generate and preserve provenance records identifying how artifacts were built and verified. | SLSA, in-toto |
| NIST SSDF PO.1, PS.2 | Artifact admission policies | Prevent unapproved or unverified model and software artifacts from proceeding to deployment. | Kyverno, Open Policy Agent |
Artifact integrity and model security are related but distinct. Signature verification establishes properties of the artifact and its origin, while behavioral evaluations and vulnerability scanning assess different security risks.
Expected Security Evidence
- Inventory of approved AI models and artifact versions.
- Model provenance and source verification records.
- Model checksum and signature verification results.
- Model file security scanning reports.
- Software dependency vulnerability reports.
- Model and software component inventories.
- Build provenance and release attestations.
- Records of approved and rejected model artifacts.
- Deployment policy enforcement results.
- Traceable relationships between model artifacts and application releases.
These artifacts provide evidence that AI model integrity and provenance are evaluated before deployment.
Verified model records should be retained for post-deployment inventory management, vulnerability analysis, and incident investigation.