• 1: Code and Prebuild
• 1.1: AI-Assisted Code Development
• 1.2: AI Agent Identity and Permissions
• 1.3: AI Application Threat Modeling
• 1.4: AI Model and Data Supply Chain
• 1.5: Prompt and Agent Configuration Security
• 2: Build and Deploy
• 2.1: AI-Generated Code Validation
• 2.2: AI Model and Artifact Integrity
• 2.3: AI Security Testing and Red Teaming
• 2.4: Secure AI Deployment
• 2.5: AI Release Evidence and Inventory
• 3: Post Deployment
• 3.1: AI Runtime Security Monitoring
• 3.2: Continuous AI Vulnerability Management
• 3.3: AI Agent Activity Monitoring
• 3.4: AI-Assisted Vulnerability Remediation
• 3.5: AI Incident Response and Recovery

1 - Code and Prebuild

Securing AI-Assisted Development and AI Application Design

Artificial intelligence is changing how software is designed and developed. AI coding assistants generate application code, autonomous agents modify repositories and dependencies, and AI models are increasingly incorporated into software applications. While these capabilities accelerate development, they introduce security risks that traditional development practices may not fully address.

Phase 1: Code and Prebuild focuses on establishing security controls before code, AI models, prompts, datasets, or autonomous agents enter the CI/CD delivery pipeline.

This phase extends established secure development practices to address AI-generated code, agent permissions, AI application threat modeling, model and data supply chain security, and the protection of AI configurations.

The objective is to identify and reduce security risks early in the software development lifecycle, before vulnerabilities or unsafe configurations are introduced into build and deployment processes.

Code and Prebuild Outcomes

By implementing these practices, development teams establish a security foundation for AI-assisted software development and AI application design.

Expected outcomes include:

• AI-generated code is subject to established secure development practices.
• Autonomous development agents operate with controlled identities and permissions.
• AI-specific application threats are identified before implementation.
• AI models, datasets, and dependencies are evaluated for security and provenance.
• Prompts, agent instructions, and configurations are protected through version control and review.
• Security requirements are documented for validation during build and deployment.

These outcomes prepare development artifacts for the automated security validation activities described in Phase 2: Build and Deploy.

Why AI Security Begins Before the Build

Traditional secure development practices focus on source code quality, dependency management, secrets protection, and secure application design. AI-assisted development introduces additional considerations. This phase will cover:

1.1 - AI-Assisted Code Development

AI-Assisted Code Development - Code and Prebuild

Securing AI-Generated Code During Software Development

Artificial intelligence is increasingly used to generate application code, recommend dependencies, create configuration files, and automate software development tasks. While AI-assisted development can accelerate delivery, generated code may introduce security vulnerabilities, insecure coding patterns, outdated dependencies, exposed credentials, or licensing concerns.

During Code and Prebuild, organizations should apply established secure development practices to all AI-generated code and changes. AI coding assistants and autonomous development agents should operate within defined security boundaries, and their output should be subject to the same security requirements as human-written code.

This activity focuses on identifying and preventing security vulnerabilities before AI-generated changes enter trusted source code repositories or proceed through the CI/CD build pipeline.

The following activities map NIST Secure Software Development Framework (SSDF), the AI SSDF Profile (NIST SP 800-218A), and OWASP security guidance to actionable CI/CD practices and suggested open-source tools. The tools are illustrative and are not mandated by these frameworks.

AI-Assisted Code Development

Objective: Ensure AI-generated code is reviewed, tested, and validated for security vulnerabilities, unsafe dependencies, exposed secrets, and licensing risks before being incorporated into the software delivery pipeline.

The NIST Secure Software Development Framework (SSDF) establishes secure coding, code review, vulnerability analysis, and software verification practices that apply to both human-written and AI-generated software.

NIST SP 800-218A, the AI SSDF Profile, extends these practices to address additional security considerations associated with AI development.

AI-generated code should not be treated as inherently secure or trusted simply because it was produced by an automated system. Development teams remain responsible for verifying that generated code meets their security requirements.

For DevOps teams, AI-generated changes should be incorporated into existing source code management, peer review, static analysis, dependency scanning, and vulnerability management workflows. Automated security checks should identify potential weaknesses before code is merged, while policies should prevent AI agents from bypassing required reviews or security gates.

Particular attention should be given to generated dependencies, insecure API usage, authentication and authorization logic, secrets handling, and changes to security-sensitive configurations.

AI-assisted development should accelerate coding activities without weakening established secure software development controls.

AI Security Requirements and Open-Source Tooling

Framework references identify relevant security practices. Tool selection and implementation depend on the organization’s development environment and security requirements.

Expected Security Evidence

• Documented secure coding requirements applicable to AI-generated code.
• Records of code reviews and approvals for AI-generated changes.
• Static application security testing results.
• Dependency vulnerability scanning reports.
• Secrets detection results and remediation records.
• Automated test execution and validation results.
• Software component inventories identifying introduced dependencies.
• License analysis reports for applicable third-party software.
• Records of security policy enforcement and rejected changes.
• Traceable pull requests linking generated changes to review and validation outcomes.

These artifacts provide evidence that AI-assisted development follows established secure software development practices and that generated code is evaluated before entering the build and deployment process.

Security findings identified during Phase 1 should be resolved or appropriately documented before the affected changes proceed to Phase 2: Build and Deploy.

Additional Resources

• NIST Secure Software Development Framework (SP 800-218)
• NIST AI SSDF Profile (SP 800-218A)
• OWASP Top 10 for LLM Applications
• OpenSSF Best Practices
• OpenSSF Security Tooling

1.2 - AI Agent Identity and Permissions

AI Agent Identity and Permissions - Phase 1: Code and Prebuild

Artificial intelligence agents are increasingly used to generate code, modify repositories, update dependencies, execute commands, and automate software development activities. Unlike traditional coding assistants, autonomous agents may perform actions directly within development environments and interact with external tools and services.

These capabilities introduce security risks when agents are granted excessive permissions, unrestricted tool access, or credentials that provide access to sensitive resources.

During Code and Prebuild, organizations should establish identity, authentication, authorization, and access control requirements for AI agents before they are integrated into development workflows.

This phase focuses on preventing unauthorized agent activity, protecting credentials, and ensuring that AI agents operate within explicitly defined security boundaries.

The following activities map NIST Secure Software Development Framework (SSDF) practices and OWASP AI security guidance to actionable CI/CD practices and suggested open-source tools. The tools are illustrative and are not mandated by these frameworks.

AI Agent Identity and Permissions

Objective: Ensure autonomous AI agents operate with verified identities, least-privilege permissions, controlled access to development resources, and appropriate authorization for security-sensitive actions.

The NIST Secure Software Development Framework (SSDF) emphasizes protecting development environments, securing software development processes, and preventing unauthorized changes to software artifacts.

The OWASP Top 10 for LLM Applications identifies excessive agency as a risk when AI systems are granted unnecessary functionality, permissions, or autonomy.

For DevOps teams, AI agent access should be treated similarly to other automated service identities. Agents should use dedicated identities where supported, receive only the permissions required for their assigned tasks, and operate within controlled execution environments.

Agent-generated instructions should never replace independent access control decisions. Permissions must be enforced by repository platforms, identity services, execution environments, and other trusted systems.

AI Security Requirements and Open-Source Tooling

Framework references identify relevant security practices and risk categories. Specific agent access controls are implementation recommendations rather than requirements explicitly prescribed for agents by every referenced framework.

Expected Security Evidence

• Documented AI agent identities and assigned responsibilities.
• Defined access control policies for repositories, tools, and services.
• Records demonstrating least-privilege permissions.
• Credential management and secrets protection configurations.
• Approved agent tool and command allowlists.
• Security policies for agent execution environments.
• Records of required reviews and approvals for agent-generated changes.
• Agent activity logs and security-relevant execution records.
• Evidence of authentication and authorization policy enforcement.

These artifacts provide evidence that autonomous AI agents are operating within controlled security boundaries and that development teams retain accountability for agent-initiated actions.

Agent security requirements established during Phase 1 should be validated during Phase 2: Build and Deploy and monitored throughout Phase 3: Post Deploy.

Additional Resources

• NIST Secure Software Development Framework
• NIST AI SSDF Profile
• OWASP Top 10 for LLM Applications
• OWASP Agentic AI Security
• SPIFFE and SPIRE

1.3 - AI Application Threat Modeling

AI Application Threat Modeling - Phase 1: Code and Prebuild

Artificial intelligence applications introduce security risks that extend beyond those found in traditional software architectures. Applications incorporating large language models (LLMs), retrieval-augmented generation (RAG), autonomous agents, external data sources, and tool integrations create new attack surfaces and trust relationships.

These systems may process untrusted prompts, retrieve external content, invoke privileged tools, and generate outputs that influence application behavior.

During Code and Prebuild, organizations should perform AI-specific threat modeling to identify potential vulnerabilities before application components are implemented or integrated into the CI/CD pipeline.

This phase focuses on identifying attack paths, evaluating security boundaries, and establishing controls to prevent unauthorized model behavior, sensitive information disclosure, and unsafe agent actions.

The following activities map NIST security guidance and OWASP AI risk categories to actionable threat modeling practices and suggested open-source tools.

AI Application Threat Modeling

Objective: Identify and document AI-specific attack surfaces, trust boundaries, potential threats, and appropriate security controls before developing or deploying AI-enabled applications.

The NIST Secure Software Development Framework (SSDF) emphasizes integrating security requirements and risk analysis into software design and development activities.

The NIST AI Risk Management Framework provides guidance for identifying, measuring, and managing risks associated with AI systems.

The OWASP Top 10 for LLM Applications identifies risks including prompt injection, sensitive information disclosure, supply chain vulnerabilities, improper output handling, and excessive agency.

For DevOps teams, threat modeling should be integrated into application architecture reviews, development planning, and security requirements definition.

AI components should be evaluated as part of the complete application architecture rather than as isolated models.

Threat models should identify how users, models, agents, retrieved information, and external services interact and where trust boundaries must be enforced.

AI Security Requirements and Open-Source Tooling

Threat modeling tools support risk identification and documentation. Testing tools such as Promptfoo and Garak can help validate selected threats but do not replace architectural threat modeling.

Expected Security Evidence

• Documented AI application threat models.
• Architecture diagrams identifying AI components and external services.
• Data flow diagrams showing model inputs, outputs, and tool interactions.
• Identified trust boundaries and security-sensitive integration points.
• Prompt injection and agent misuse risk assessments.
• Sensitive information exposure assessments.
• Documented AI security risks and assigned risk owners.
• Security requirements and mitigation plans derived from identified threats.
• Records of architecture reviews and threat model updates.

These artifacts provide evidence that AI-specific security risks were considered during application planning and design.

Threat models developed during Phase 1 should inform the automated AI security testing and release validation activities performed during Phase 2: Build and Deploy.

Additional Resources

• NIST Secure Software Development Framework
• NIST AI SSDF Profile
• NIST AI Risk Management Framework
• OWASP Top 10 for LLM Applications
• OWASP Threat Dragon
• MITRE ATLAS

1.4 - AI Model and Data Supply Chain

AI Model and Data Supply Chain - Phase 1: Code and Prebuild

AI-enabled applications increasingly depend on externally sourced models, training datasets, machine learning frameworks, model repositories, and third-party software components.

These dependencies introduce security risks beyond traditional software package management. AI models may originate from untrusted sources, datasets may contain malicious or manipulated content, and model files may include unsafe serialization formats or dependencies.

During Code and Prebuild, organizations should establish processes for evaluating the provenance, integrity, licensing, and security of AI models and data before incorporating them into applications.

This phase focuses on identifying supply chain risks early, maintaining visibility into AI dependencies, and establishing trusted sources for AI components.

The following activities map NIST Secure Software Development Framework (SSDF), the AI SSDF Profile, and OWASP AI security guidance to actionable CI/CD practices and suggested open-source tools.

AI Model and Data Supply Chain

Objective: Ensure AI models, datasets, frameworks, and third-party dependencies are evaluated for provenance, integrity, vulnerabilities, licensing, and supply chain risks before entering the software delivery pipeline.

The NIST Secure Software Development Framework (SSDF) establishes practices for identifying, evaluating, and securing third-party software components.

The NIST AI SSDF Profile (SP 800-218A) extends secure development guidance to address additional security considerations associated with AI model development and integration.

The OWASP Top 10 for LLM Applications identifies AI supply chain vulnerabilities and data and model poisoning as significant risks.

For DevOps teams, AI model and data dependencies should be evaluated before being introduced into trusted development environments.

This includes verifying component sources, reviewing applicable licenses, identifying known vulnerabilities, and documenting dependencies between models, datasets, frameworks, and application components.

Organizations should also establish requirements for maintaining accurate inventories of AI assets so they can be tracked throughout the build, deployment, and operational lifecycle.

AI Security Requirements and Open-Source Tooling

These tools address different aspects of AI supply chain security. Traditional SBOM tools do not necessarily capture complete model, dataset, or prompt inventories. Model provenance, dataset security, and licensing may require additional records and specialized validation.

Expected Security Evidence

• Inventory of approved AI models and associated versions.
• Records identifying model sources, repositories, and provenance.
• Model integrity verification results and available signatures.
• Dataset source records and integrity assessments.
• Documented assessments of data and model poisoning risks.
• Software dependency vulnerability scanning results.
• Model file security scan results.
• Third-party component license and usage reviews.
• AI component inventories and applicable SBOMs.
• Records of rejected or unapproved AI components.
• Traceable relationships between models, datasets, dependencies, and application development artifacts.

These artifacts provide evidence that AI models and supporting dependencies have been evaluated before entering the software delivery process.

Supply chain records created during Phase 1 should support model verification and artifact integrity controls in Phase 2: Build and Deploy, as well as continuous vulnerability management during Phase 3: Post Deploy.

Additional Resources

• NIST Secure Software Development Framework
• NIST AI SSDF Profile
• OWASP Top 10 for LLM Applications
• OpenSSF Model Signing
• CycloneDX
• OpenSSF

1.5 - Prompt and Agent Configuration Security

Prompt and Agent Configuration Security - Phase 1: Code and Prebuild

AI-enabled applications rely on system prompts, agent instructions, tool definitions, and configuration files to determine how models interact with users, applications, and external services.

These artifacts can directly influence application behavior, data access, tool execution, and security boundaries.

Unlike traditional source code, prompts and agent configurations may be modified independently of application binaries. Unauthorized or poorly reviewed changes can introduce security risks even when the underlying application code remains unchanged.

During Code and Prebuild, organizations should treat security-sensitive prompts, agent instructions, and configurations as managed development artifacts.

This phase focuses on protecting AI configurations against unauthorized modification, establishing secure review processes, and ensuring that agent capabilities remain aligned with approved security policies.

The following activities map NIST security guidance and OWASP AI risk categories to actionable CI/CD practices and suggested open-source tools.

Prompt and Agent Configuration Security

Objective: Ensure system prompts, agent instructions, tool configurations, and AI application settings are version-controlled, reviewed, protected against unauthorized changes, and aligned with established security requirements.

The NIST Secure Software Development Framework (SSDF) establishes practices for protecting software artifacts, controlling changes, and integrating security requirements into development workflows.

The OWASP Top 10 for LLM Applications identifies prompt injection, system prompt leakage, and excessive agency as security concerns affecting AI-enabled applications.

For DevOps teams, prompts and agent configurations should be managed using established source control and change management practices.

Security-sensitive changes should be reviewed before acceptance, and automated policies should prevent unauthorized modifications to agent permissions, tool access, and execution settings.

System prompts should not be treated as a security boundary. Authorization must be enforced by trusted application and infrastructure controls independently of model instructions.

AI Security Requirements and Open-Source Tooling

The referenced frameworks provide relevant software security practices and AI risk categories. The implementation recommendations translate those practices into CI/CD controls for prompts, agent instructions, and configuration artifacts.

Expected Security Evidence

• Version-controlled system prompts and agent instructions.
• Approved tool definitions and agent configuration records.
• Security review and approval records for configuration changes.
• Documented policies restricting agent permissions and tool access.
• Configuration validation and policy enforcement results.
• Secrets scanning results for prompts and configuration files.
• Prompt injection testing results and identified security findings.
• Records of unauthorized or rejected configuration modifications.
• Traceable configuration versions associated with application changes.
• Documented procedures for updating and restoring approved AI configurations.

These artifacts provide evidence that AI instructions and configurations are managed through controlled development processes.

Configuration controls established during Phase 1 should support security testing and release verification during Phase 2: Build and Deploy.

After deployment, changes to AI instructions, tools, and permissions should remain subject to authorization, monitoring, and audit requirements.

Additional Resources

• NIST Secure Software Development Framework
• NIST AI SSDF Profile
• OWASP Top 10 for LLM Applications
• OWASP Generative AI Security Project
• Open Policy Agent
• Promptfoo

2 - Build and Deploy

Validating AI Artifacts and Securing AI-Enabled Software Delivery

Artificial intelligence introduces new security considerations into the build and deployment process. AI-generated code, machine learning models, prompts, agent configurations, and supporting dependencies must be validated before they are incorporated into production software.

Phase 2: Build and Deploy focuses on ensuring that AI-generated code and AI application components are securely built, tested, verified, and authorized for deployment.

This phase extends traditional CI/CD security practices to address AI-specific vulnerabilities, model integrity, adversarial testing, agent permissions, and the traceability of AI-related software artifacts.

The objective is to establish automated security controls that verify the integrity and security of AI-enabled software before it reaches production.

Phase 2 Outcomes

By implementing these practices, DevOps teams establish automated security controls for validating and deploying AI-enabled software.

Expected outcomes include:

• AI-generated code is validated using established secure development and testing practices.
• AI models and supporting artifacts are verified for integrity and provenance.
• AI applications are tested for prompt injection, unsafe behavior, and unauthorized tool execution.
• Autonomous agents and AI services operate within defined deployment permissions.
• Software releases include traceable AI artifacts, dependencies, and security evaluation results.
• Deployment authorization and security policies are consistently enforced.
• Security evidence is available for ongoing vulnerability management and compliance assessments.

These outcomes help ensure that AI-enabled applications enter production with verified artifacts, tested security controls, and traceable release information.

Related Security Frameworks

The following frameworks provide guidance relevant to Phase 2 AI security activities.

These frameworks and projects provide guidance and technical capabilities that can support AI security controls. They do not all establish mandatory regulatory compliance requirements.

Integrating AI Security into the CI/CD Pipeline

AI security controls should be incorporated into existing CI/CD automation rather than implemented as disconnected manual processes.

This section will cover a typical AI-enabled software delivery workflow to include:

2.1 - AI-Generated Code Validation

AI-Generated Code Validation - Phase 2: Build and Deploy

Artificial intelligence is increasingly used to generate source code, update dependencies, modify configurations, and create pull requests. These changes can accelerate software delivery but may introduce vulnerabilities, insecure coding patterns, compatibility problems, and unintended functionality.

During Build and Deploy, organizations should independently validate AI-generated changes using established security testing and software quality controls.

AI-generated code should not be trusted solely because an AI assistant or autonomous agent reports that its changes are secure or functional.

This activity focuses on incorporating automated scanning, testing, and release authorization into CI/CD workflows to ensure AI-generated code meets established security requirements before deployment.

AI-Generated Code Validation

Objective: Ensure AI-generated code and configuration changes are independently tested, reviewed, and validated for security vulnerabilities, vulnerable dependencies, exposed secrets, and unintended functionality before software is released.

The NIST Secure Software Development Framework (SSDF) establishes practices for reviewing, analyzing, and testing software to identify vulnerabilities before release.

The NIST AI SSDF Profile (SP 800-218A) extends secure development guidance to address AI-related development considerations.

For DevOps teams, automated CI/CD validation provides an independent security control between AI-generated changes and production deployment.

Security gates should prevent code from progressing when required validation fails. Human review or authorized policy decisions should determine whether changes are accepted, rejected, or returned for remediation.

AI Security Requirements and Open-Source Tooling

These mappings apply established secure software development practices to AI-generated changes. AI-generated code remains subject to the same security requirements as human-written code.

Expected Security Evidence

• Automated static application security testing reports.
• Dependency vulnerability scanning results.
• Secrets detection and remediation records.
• Unit, integration, and regression test results.
• Build and dependency compatibility validation logs.
• Records of rejected or failed AI-generated changes.
• Code review and release approval records.
• Security policy enforcement results.
• Artifact verification and provenance records.
• Traceable pull requests linking AI-generated changes to validation results.

These artifacts demonstrate that AI-generated changes have been independently validated before entering production.

Security findings should be resolved, rejected, or formally accepted through an authorized risk management process before deployment.

Additional Resources

• NIST Secure Software Development Framework
• NIST AI SSDF Profile
• OpenSSF
• OWASP ZAP
• Open Policy Agent

2.2 - AI Model and Artifact Integrity

AI Model and Artifact Integrity - Phase 2: Build and Deploy

AI-enabled applications increasingly incorporate pretrained models, fine-tuned models, embeddings, model adapters, and supporting software artifacts obtained from external repositories.

These artifacts introduce supply chain security risks, including compromised models, unauthorized modifications, unsafe serialization formats, and untrusted dependencies.

During Build and Deploy, organizations should verify that AI models and supporting artifacts originate from approved sources and have not been modified unexpectedly.

This activity extends established software supply chain security practices to AI models, associated dependencies, and release artifacts.

AI Model and Artifact Integrity

Objective: Ensure AI models and supporting software artifacts are verified for provenance, integrity, authenticity, and known security risks before packaging or deployment.

The NIST Secure Software Development Framework (SSDF) establishes practices for protecting software artifacts and verifying software integrity.

The NIST AI SSDF Profile (SP 800-218A) addresses additional security considerations associated with AI model development and integration.

The OpenSSF Model Signing project provides open-source capabilities for signing and verifying AI model artifacts.

For DevOps teams, model verification should become an automated CI/CD activity similar to container signature verification, dependency scanning, and software artifact validation.

Model signatures and checksums help establish artifact integrity, but do not independently prove that a model is free of malicious behavior or vulnerabilities. Additional security assessments remain necessary.

AI Security Requirements and Open-Source Tooling

Artifact integrity and model security are related but distinct. Signature verification establishes properties of the artifact and its origin, while behavioral evaluations and vulnerability scanning assess different security risks.

Expected Security Evidence

• Inventory of approved AI models and artifact versions.
• Model provenance and source verification records.
• Model checksum and signature verification results.
• Model file security scanning reports.
• Software dependency vulnerability reports.
• Model and software component inventories.
• Build provenance and release attestations.
• Records of approved and rejected model artifacts.
• Deployment policy enforcement results.
• Traceable relationships between model artifacts and application releases.

These artifacts provide evidence that AI model integrity and provenance are evaluated before deployment.

Verified model records should be retained for post-deployment inventory management, vulnerability analysis, and incident investigation.

Additional Resources

• NIST Secure Software Development Framework
• NIST AI SSDF Profile
• OpenSSF Model Signing
• SLSA
• Sigstore
• CycloneDX

2.3 - AI Security Testing and Red Teaming

AI Security Testing and Red Teaming - Phase 2: Build and Deploy

AI-enabled applications introduce security risks that traditional application security testing may not fully identify.

Large language models (LLMs), retrieval-augmented generation (RAG) systems, and autonomous AI agents process dynamic inputs, interact with external information, and may invoke tools or execute actions.

These capabilities create attack surfaces involving prompt injection, sensitive information disclosure, unauthorized tool execution, insecure output handling, and excessive agency.

During Build and Deploy, organizations should incorporate AI-specific security evaluations and adversarial testing into CI/CD workflows.

This activity focuses on identifying unsafe AI application behavior before production deployment and establishing repeatable security testing practices.

AI Security Testing and Red Teaming

Objective: Identify AI-specific vulnerabilities and unsafe behavior through automated security evaluations, adversarial testing, and independently enforced release acceptance criteria.

The NIST Secure Software Development Framework (SSDF) establishes security testing and vulnerability identification practices for software releases.

The NIST AI Risk Management Framework emphasizes evaluating and managing risks throughout the AI system lifecycle.

The OWASP Top 10 for LLM Applications identifies common AI application security risks, including prompt injection, sensitive information disclosure, improper output handling, and excessive agency.

For DevOps teams, AI security evaluations should be automated wherever practical and incorporated into existing build and release validation processes.

Security testing should evaluate not only the model’s responses but also whether the application enforces access controls, protects sensitive information, and prevents unauthorized tool actions.

Because AI behavior may be nondeterministic, teams should define repeatable test scenarios, execution parameters, evaluation criteria, and acceptable failure thresholds.

AI Security Requirements and Open-Source Tooling

AI security testing tools have different capabilities and coverage. No single testing tool can establish that an AI application is free of vulnerabilities or unsafe behaviors.

Expected Security Evidence

• AI application security test plans and acceptance criteria.
• Prompt injection testing results.
• Sensitive information disclosure evaluation reports.
• Agent permission and tool execution test results.
• RAG and retrieval security testing records.
• Resource consumption and abuse testing results.
• Security regression test results.
• Records of identified vulnerabilities and remediation activities.
• Automated security gate results.
• Approved exceptions and documented residual risks.
• Traceable AI evaluation results associated with application releases.

These artifacts demonstrate that AI applications have undergone defined security evaluations before deployment.

Security testing should be repeated when significant changes occur to models, prompts, agent permissions, retrieval sources, or connected tools.

Additional Resources

• NIST Secure Software Development Framework
• NIST AI SSDF Profile
• NIST AI Risk Management Framework
• OWASP Top 10 for LLM Applications
• MITRE ATLAS
• Inspect AI

2.4 - Secure AI Deployment

Secure AI Deployment - Phase 2: Build and Deploy

AI-enabled applications often depend on models, external APIs, vector databases, retrieval services, and autonomous agents that interact with other applications and infrastructure.

These integrations introduce security risks when deployed applications or agents receive excessive permissions, unrestricted network access, sensitive credentials, or unauthorized tool capabilities.

During Build and Deploy, organizations should ensure AI applications and autonomous agents are deployed within controlled environments that enforce access policies, workload isolation, secure configuration, and release authorization.

This activity extends established CI/CD deployment security practices to address the operational capabilities and security requirements of AI-enabled applications.

Secure AI Deployment

Objective: Ensure AI applications, models, and autonomous agents are deployed using verified artifacts, controlled identities, least-privilege permissions, protected credentials, and independently enforced security policies.

The NIST Secure Software Development Framework (SSDF) establishes practices for protecting software development environments and ensuring the integrity of software releases.

The OWASP Top 10 for LLM Applications identifies excessive agency, prompt injection, and improper output handling as risks that can affect deployed AI applications.

For DevOps teams, secure deployment should ensure that AI services and agents operate only within approved access and execution boundaries.

AI-generated instructions must not independently grant permissions or authorize privileged operations. Access decisions should be enforced by trusted application, identity, and infrastructure controls.

AI Security Requirements and Open-Source Tooling

These controls apply established software security and infrastructure practices to AI-enabled applications. Deployment authorization and infrastructure security controls should operate independently of model-generated decisions.

Expected Security Evidence

• Approved AI deployment configurations.
• Workload identity and authentication records.
• Agent permission and authorization policies.
• Secrets management and credential access configurations.
• Workload isolation and network security policies.
• Verified container and artifact signatures.
• Deployment policy enforcement records.
• Release approval and change authorization evidence.
• Deployment configuration and version history.
• Records of rejected or unauthorized deployment attempts.

These artifacts provide evidence that AI applications and agents are deployed within controlled environments and operate under defined security restrictions.

Deployment evidence should be retained for continuous security monitoring, vulnerability management, and incident response during Phase 3: Post Deploy.

Additional Resources

• NIST Secure Software Development Framework
• NIST AI SSDF Profile
• OWASP Top 10 for LLM Applications
• Open Policy Agent
• SPIFFE and SPIRE
• Kubernetes Security

2.5 - AI Release Evidence and Inventory

AI Release Evidence and Inventory - Phase 2: Build and Deploy

AI-enabled software releases may include more than traditional source code, libraries, and container images.

Applications can depend on AI models, prompts, agent configurations, retrieval components, external services, and supporting datasets that influence their behavior.

Without an accurate inventory of these components, organizations may struggle to identify which AI assets are deployed, determine which systems are affected by security findings, or reproduce the configuration of a previous release.

During Build and Deploy, organizations should create and retain inventories and security evidence that associate AI components with their application releases and deployment records.

This activity focuses on establishing traceability across the AI software supply chain and ensuring security evidence remains available after deployment.

AI Release Evidence and Inventory

Objective: Maintain accurate, traceable records of AI models, prompts, agent configurations, software dependencies, security evaluations, and deployment artifacts associated with each software release.

The NIST Secure Software Development Framework (SSDF) establishes practices for preserving software integrity, tracking software components, and generating evidence associated with secure development.

The NIST AI SSDF Profile (SP 800-218A) extends secure development guidance to address AI model development and lifecycle considerations.

For DevOps teams, release inventories should extend beyond traditional Software Bills of Materials (SBOMs) to include relevant AI assets and their relationships to deployed applications.

An SBOM identifies software components and dependencies, but may not fully capture models, datasets, prompts, and agent configurations. Additional AI-specific inventories or metadata may therefore be necessary.

The resulting release evidence should enable teams to determine what was built, what was tested, what was approved, and what was deployed.

AI Security Requirements and Open-Source Tooling

Release inventories may require multiple complementary formats and tools. SBOMs, model metadata, provenance attestations, security evaluation results, and deployment records serve different purposes and should be linked using stable identifiers wherever practical.

Expected Security Evidence

• Software Bills of Materials for applicable software artifacts.
• AI model inventories identifying model names, versions, and sources.
• Model integrity verification records.
• Approved prompt and agent configuration versions.
• AI dependency and component relationship records.
• Build provenance and artifact attestations.
• AI security testing and evaluation reports.
• Vulnerability scanning and remediation records.
• Release approvals and authorization decisions.
• Deployment records identifying target environments and application versions.
• Traceable links between AI artifacts, application releases, and deployed systems.
• Retained security evidence supporting investigation, audits, and vulnerability management.

These artifacts provide a record of the software and AI components included in each release and the security controls applied before deployment.

Release inventories should remain available after deployment so newly discovered vulnerabilities can be correlated with affected software components, AI models, applications, and environments.

This information provides the foundation for continuous vulnerability management, affected-system identification, and incident response during Phase 3: Post Deploy.

Additional Resources

• NIST Secure Software Development Framework
• NIST AI SSDF Profile
• CycloneDX
• SLSA
• Sigstore
• GUAC
• OpenSSF Model Signing

3 - Post Deployment

Continuous AI Security Monitoring and Controlled Remediation

Security does not stop when an AI-enabled application reaches production. New vulnerabilities are discovered, AI models and dependencies change, autonomous agents interact with external systems, and emerging threats can affect applications long after their initial deployment.

Phase 3: Post Deploy focuses on maintaining continuous security visibility into deployed AI applications, models, agents, and their supporting software components.

This phase extends traditional vulnerability management and runtime security practices to address AI-specific threats, including prompt injection, unauthorized agent activity, sensitive information disclosure, model compromise, and unsafe automated actions.

It also explores how artificial intelligence can support vulnerability analysis and remediation while maintaining independent validation, human oversight, and auditable security controls.

The objective is to ensure that deployed AI-enabled software remains secure throughout its operational lifecycle.

Why AI Security Must Continue After Deployment

Traditional post-deployment security activities focus on monitoring applications, identifying vulnerabilities, detecting suspicious behavior, and applying security updates.

AI-enabled applications introduce additional challenges:

• Changing AI Threats: Newly discovered attack techniques can affect deployed AI applications without requiring changes to their underlying source code.
• Autonomous Agent Activity: AI agents may execute commands, access APIs, modify resources, or interact with external systems during production operations.
• Model and Dependency Vulnerabilities: Deployed AI services depend on models, frameworks, libraries, containers, and supporting infrastructure that may develop new security exposures.
• Prompt Injection and Data Exposure: Malicious inputs and untrusted external content can manipulate model behavior or expose sensitive information.
• Limited Deployment Visibility: Organizations may lack an accurate inventory of which AI models, prompts, dependencies, and configurations are running in production.
• AI-Assisted Remediation Risks: AI-generated security fixes may introduce compatibility issues, unintended changes, or new vulnerabilities if applied without validation.

Addressing these challenges requires continuous security monitoring, accurate deployed asset inventories, and controlled remediation workflows.

Phase 3: Post Deployment Outcomes

By implementing these practices, DevOps and security teams establish continuous security oversight for deployed AI-enabled software.

Expected outcomes include:

• Deployed AI applications are monitored for suspicious behavior, unauthorized access, and emerging threats.
• AI agents operate with observable and auditable execution histories.
• Newly disclosed software vulnerabilities are correlated with affected production applications and dependencies.
• AI-specific behavioral risks are assessed alongside traditional software vulnerabilities.
• AI-assisted remediation changes are independently tested and approved before release.
• Compromised models, agents, and applications can be isolated, rolled back, or recovered.
• Security evidence supports incident investigation, vulnerability management, and compliance assessments.
• Production security findings provide feedback into earlier CI/CD security activities.

These outcomes support a continuous security model in which monitoring, vulnerability identification, remediation, and verification extend throughout the operational lifecycle.

Related Security Frameworks

The following frameworks provide guidance relevant to Phase 3 AI security activities.

These frameworks provide guidance for implementing and managing AI security controls. They do not all establish mandatory regulatory compliance requirements.

Integrating AI Security into the CI/CD Pipeline

AI security controls should be incorporated into existing CI/CD automation rather than implemented as disconnected manual processes.

This section will cover a typical AI-enabled software delivery workflow to include:

3.1 - AI Runtime Security Monitoring

AI Runtime Security Monitoring - Phase 3: Post Deploy

Artificial intelligence applications introduce runtime behaviors that traditional application security monitoring may not fully capture. Large language models (LLMs), retrieval-augmented generation (RAG) systems, and autonomous agents interact with users, external data sources, APIs, and supporting services in ways that can create additional security risks.

These risks include prompt injection, sensitive information disclosure, unauthorized tool execution, model manipulation, and resource abuse.

During Post Deploy, organizations should continuously monitor AI applications to identify suspicious activity, detect security policy violations, and respond to emerging threats.

This activity extends established runtime security monitoring practices to include AI-specific behavior, model interactions, and application security events.

AI Runtime Security Monitoring

Objective: Detect and investigate suspicious AI application behavior, unauthorized access, prompt injection attempts, sensitive information exposure, and runtime security policy violations.

The NIST AI Risk Management Framework emphasizes ongoing measurement, monitoring, and management of risks associated with AI systems.

The NIST Cybersecurity Framework 2.0 provides guidance for identifying, detecting, responding to, and recovering from cybersecurity threats.

The OWASP Top 10 for LLM Applications identifies AI-specific threats including prompt injection, sensitive information disclosure, excessive agency, and unbounded consumption.

For DevOps teams, runtime monitoring should combine application telemetry, infrastructure security events, AI interaction traces, and independently enforced security policies.

Monitoring should collect sufficient information to investigate suspicious behavior while protecting confidential prompts, user information, credentials, and sensitive model outputs.

AI-generated explanations of suspicious activity should not replace independently collected security evidence.

AI Security Requirements and Open-Source Tooling

Observability tools collect and analyze runtime information but do not automatically detect every AI-specific attack. Prompt injection testing tools primarily support security evaluations and require additional integration for continuous monitoring.

Expected Security Evidence

• AI application runtime monitoring configurations.
• AI service traces, metrics, and security event records.
• Records of detected suspicious input or application behavior.
• Unauthorized tool execution and access violation alerts.
• Sensitive information exposure detection results.
• Resource usage and abnormal consumption monitoring reports.
• Infrastructure and container runtime security findings.
• Correlated application and security event records.
• Security incident investigation records.
• Runtime policy enforcement and alert response evidence.

These artifacts demonstrate that deployed AI applications are subject to continuous security monitoring and that security-relevant events can be investigated.

Monitoring findings should feed into vulnerability management, incident response, and the security requirements applied during earlier CI/CD phases.

Additional Resources

• NIST AI Risk Management Framework
• NIST Cybersecurity Framework 2.0
• OWASP Top 10 for LLM Applications
• OpenTelemetry
• Falco
• MITRE ATLAS

3.2 - Continuous AI Vulnerability Management

Continuous AI Vulnerability Management - Phase 3: Post Deploy

AI-enabled applications depend on software frameworks, model-serving infrastructure, libraries, containers, APIs, and supporting services that may become vulnerable after deployment.

A software component that passes security testing during development may later be affected by a newly disclosed Common Vulnerabilities and Exposures (CVE) record.

Traditional pre-deployment scanning cannot detect vulnerabilities that become known after software has already been released.

AI applications also introduce behavioral security risks, including prompt injection, excessive agent permissions, sensitive information disclosure, and manipulation of model outputs. These weaknesses may not have CVE identifiers and cannot always be addressed through traditional software patching.

During Post Deploy, organizations should continuously monitor both newly disclosed software vulnerabilities and AI-specific behavioral security risks.

This activity focuses on maintaining deployed software visibility, identifying affected environments, assessing AI-specific threats, prioritizing remediation, and verifying that corrective actions reach affected production systems.

Continuous AI Vulnerability Management

Objective: Continuously identify, assess, prioritize, and remediate software vulnerabilities and AI-specific behavioral security weaknesses affecting deployed AI applications, dependencies, agents, and supporting infrastructure.

The NIST Secure Software Development Framework (SSDF) establishes practices for identifying vulnerabilities in released software, assessing their impact, and implementing appropriate responses.

The NIST AI SSDF Profile (SP 800-218A) extends secure development guidance to address AI model development and integration.

The NIST Cybersecurity Framework 2.0 provides guidance for identifying technology assets, managing vulnerabilities, assessing risks, and implementing appropriate responses.

The NIST AI Risk Management Framework provides guidance for identifying, measuring, monitoring, and managing risks associated with AI systems.

The OWASP Top 10 for LLM Applications identifies AI-specific security risks, including prompt injection, sensitive information disclosure, improper output handling, and excessive agency.

For DevOps teams, continuous AI security requires two complementary practices:

1. Software Vulnerability Management: Identify known software vulnerabilities, correlate findings with deployed components, and verify remediation.
2. AI Behavioral Risk Management: Identify and mitigate AI-specific weaknesses through adversarial testing, runtime monitoring, access controls, and application security improvements.

These practices should share application inventory, ownership, and deployment information while maintaining distinct methods for assessing and remediating security findings.

Software Vulnerability Management - NIST SSDF

Software vulnerability management focuses on identifying, assessing, and remediating known vulnerabilities in deployed software components, including AI frameworks, libraries, containers, model-serving infrastructure, and supporting services.

NIST SSDF practices RV.1 and RV.2 establish processes for identifying and responding to vulnerabilities in released software.

For DevOps teams, vulnerability management must extend beyond creating Software Bills of Materials (SBOMs) and scanning release artifacts.

SBOM information should be associated with application versions and deployment metadata so teams can identify which vulnerable components are actually running, where they are deployed, and who is responsible for remediation.

Continuous monitoring of vulnerability intelligence sources enables organizations to identify vulnerabilities disclosed after deployment and determine their potential impact on production applications.

Remediation should be prioritized based on severity, known exploitation, exposure, affected production systems, and operational risk.

Software Vulnerability Management Requirements and Open-Source Tooling

Vulnerability discovery, software supply chain correlation, and deployment inventory are complementary capabilities. An SBOM scanner identifies software components and may detect known vulnerabilities, while deployment correlation is needed to determine which applications and environments are affected.

Software Vulnerability Management - Expected Security Evidence

• Inventory of deployed AI applications and supporting software components.
• SBOMs associated with released application versions.
• Deployment records identifying environments and affected assets.
• Newly disclosed CVE detection records.
• Correlations between vulnerable packages and deployed applications.
• Vulnerability severity, exploitability, and exposure assessments.
• Documented vulnerability ownership and prioritization decisions.
• Remediation tracking records and assigned corrective actions.
• Post-remediation vulnerability scanning results.
• Evidence confirming deployment of corrected software versions.

These artifacts provide evidence that software vulnerability management continues after deployment and that newly disclosed vulnerabilities can be associated with affected production systems.

Vulnerability findings should initiate appropriate remediation workflows, including the AI-assisted remediation activities described in this phase.

AI Behavioral Risk Management - OWASP and NIST AI RMF

AI-enabled applications introduce behavioral security weaknesses that may not be associated with a known CVE or vulnerable software package. These weaknesses arise from how AI systems process instructions, interact with external content, access tools, and make decisions.

Examples include prompt injection, excessive agency, sensitive information disclosure, unsafe output handling, and manipulation of retrieved information.

Unlike traditional software vulnerabilities, these issues may require changes to application architecture, trust boundaries, authorization policies, model configurations, prompts, or agent permissions rather than dependency upgrades.The OWASP Top 10 for LLM Applications identifies common AI-specific security risks, while the NIST AI Risk Management Framework provides guidance for assessing, monitoring, and managing AI risks throughout the operational lifecycle.

For DevOps teams, AI behavioral risk management should be integrated into continuous security testing, runtime monitoring, application configuration management, and incident response processes. Security findings should be associated with affected AI applications, model versions, agent configurations, and deployed environments whenever possible.

AI Behavioral Risk Management Requirements and Open-Source Tooling

OWASP identifies AI application security risk categories, while NIST AI RMF provides processes for managing AI risks. The suggested controls and tools are implementation examples rather than mandatory requirements specified by these frameworks.

AI Behavioral Risk Management - Expected Security Evidence

• AI application threat models and behavioral risk assessments.
• Prompt injection testing and evaluation results.
• Sensitive information disclosure assessments.
• Agent permission and authorization validation records.
• Security testing results for retrieval and external tool integrations.
• AI model and application evaluation reports.
• Runtime monitoring and security event records.
• Documented AI behavioral weaknesses and affected applications.
• AI security risk ownership and prioritization decisions.
• Mitigation plans and security configuration changes.
• Post-mitigation security testing and validation results.
• Records of accepted, unresolved, or escalated AI security risks.

These artifacts provide evidence that AI-specific behavioral security risks are being assessed and managed throughout the operational lifecycle. Because AI behavior may vary with model versions, prompts, retrieved content, and external integrations, security evaluations should be repeated when significant changes occur.

Coordinating Software Vulnerabilities and AI Behavioral Risks

Software vulnerability management and AI behavioral risk management should operate as complementary security processes. A deployed AI application may contain vulnerable software dependencies while also exhibiting unsafe behavior caused by excessive agent permissions, prompt injection, or insecure application design.

Both categories of findings should be linked to the affected applications and deployment environments, but their detection and remediation requirements may differ.

For DevOps and platform engineering teams, maintaining a connection between these security processes improves visibility into the complete security posture of deployed AI applications.

A common application inventory, ownership model, and deployment record can help correlate findings while preserving the distinct evaluation and remediation workflows required for each risk category.

Additional Resources

• NIST Secure Software Development Framework
• NIST AI SSDF Profile
• NIST Cybersecurity Framework 2.0
• NIST AI Risk Management Framework
• OWASP Top 10 for LLM Applications
• MITRE ATLAS
• OSV
• GUAC
• Ortelius
• CycloneDX

3.3 - AI Agent Activity Monitoring

AI Agent Activity Monitoring - Phase 3: Post Deploy

Autonomous AI agents introduce security risks because they can execute actions rather than simply generate responses. Production agents may interact with APIs, access databases, execute commands, modify files, invoke external tools, and communicate with other agents. These capabilities create security concerns when agents perform unauthorized actions, exceed assigned permissions, expose sensitive information, or respond to manipulated instructions.

During Post Deploy, organizations should continuously monitor AI agent activity to detect suspicious behavior, enforce authorization boundaries, and maintain auditable records of agent operations.

This activity focuses on establishing visibility into agent identities, tool calls, permissions, execution history, and security-sensitive actions.

AI Agent Activity Monitoring

Objective: Ensure autonomous AI agent activity is observable, attributable, and auditable, with controls capable of detecting unauthorized actions, excessive permissions, and security policy violations.

The NIST AI Risk Management Framework provides guidance for monitoring and managing AI-related risks throughout system operation.

The NIST Cybersecurity Framework 2.0 establishes practices for monitoring systems, detecting suspicious activity, and managing access-related security risks.

The OWASP Top 10 for LLM Applications identifies excessive agency as a security risk when AI systems have unnecessary functionality, permissions, or autonomy.

For DevOps teams, autonomous agents should be monitored similarly to other privileged automated services, with additional visibility into model-directed tool selection and execution.

Security controls should record which agent initiated an action, what resource was accessed, which tool was invoked, and whether the action was authorized.

Agent-generated explanations should not replace independent audit logs, access control records, and execution telemetry.

AI Security Requirements and Open-Source Tooling

Agent observability and authorization are separate controls. Monitoring records what an agent attempted or performed, while trusted authorization systems must independently determine which actions are permitted.

Expected Security Evidence

• Inventory of active AI agents and assigned identities.
• Agent authentication and authorization records.
• Agent tool invocation and execution logs.
• Records of access to protected APIs and services.
• Alerts identifying unauthorized or suspicious agent actions.
• Agent permission and policy enforcement results.
• Runtime process execution and infrastructure security events.
• Correlated agent activity and application security records.
• Records of suspended agents and revoked credentials.
• Security investigation and remediation records.
• Audit trails linking agent actions to initiating requests and execution contexts.

These artifacts demonstrate that autonomous AI agents operate under monitored and enforceable security controls.

Agent monitoring should support timely investigation of unauthorized operations while retaining sufficient evidence to establish which actions occurred and what systems were affected.

Additional Resources

• NIST AI Risk Management Framework
• NIST Cybersecurity Framework 2.0
• OWASP Excessive Agency
• OpenTelemetry
• Falco
• Open Policy Agent

3.4 - AI-Assisted Vulnerability Remediation

AI-Assisted Vulnerability Remediation - Phase 3: Post Deploy

Traditional vulnerability management tools identify vulnerable software packages and report security findings, but correcting those vulnerabilities frequently requires manual investigation, dependency analysis, code modifications, compatibility testing, and deployment coordination.

AI-assisted remediation introduces opportunities to reduce this effort by analyzing vulnerability information, identifying candidate fixes, and generating proposed software changes.

However, AI-generated remediation may introduce breaking changes, dependency conflicts, functional regressions, or new vulnerabilities.

During Post Deploy, organizations should integrate AI-assisted remediation into controlled CI/CD workflows that independently validate proposed changes before they are accepted or deployed.

This activity focuses on connecting deployed vulnerability detection with AI-assisted analysis, remediation preparation, automated validation, and authorized software delivery.

AI-Assisted Vulnerability Remediation

Objective: Accelerate vulnerability remediation through AI-assisted analysis and code generation while ensuring that proposed fixes are independently tested, reviewed, and authorized before deployment.

The NIST Secure Software Development Framework (SSDF) establishes practices for identifying and addressing vulnerabilities in released software.

The NIST AI SSDF Profile (SP 800-218A) extends secure development guidance to AI-related systems and development practices.

The NIST Cybersecurity Framework 2.0 provides guidance for managing cybersecurity risks and implementing appropriate corrective actions.

For DevOps teams, AI-assisted remediation should begin with an identified vulnerability and sufficient information about affected software components, dependency relationships, and deployment environments.

AI may analyze the vulnerability and prepare candidate updates, but independent tools and authorized reviewers should determine whether the proposed changes are acceptable.

Remediation workflows should not allow AI agents to bypass testing, change approval, or production deployment controls.

AI Security Requirements and Open-Source Tooling

The listed AI development tools can assist with code changes, but they do not independently provide a complete vulnerability remediation system. Organizations must integrate vulnerability intelligence, dependency analysis, testing, authorization, and deployment verification.

Controlled AI Remediation Workflow

AI-assisted remediation should distinguish three possible outcomes:

1. Safe Dependency Upgrade

A vulnerability has a known patched version that is compatible with the application.

The AI proposes an update, and independent build, security, and regression tests confirm the change satisfies release requirements.

The proposed fix is submitted for review and authorization before deployment.

2. Unsafe Dependency Upgrade

A patched version is available, but upgrading introduces breaking changes, dependency conflicts, or failed tests.

The proposed change is blocked or revised. The workflow records the failure and may generate a tracked issue for further investigation.

3. Ambiguous or Unresolved Vulnerability

A vulnerability cannot be safely corrected because the available evidence is insufficient, a compatible patch does not exist, or the remediation risk is unacceptable.

The workflow preserves the vulnerability finding, documents the available alternatives, and creates an issue for human investigation.

These outcomes ensure AI-assisted remediation produces auditable decisions rather than uncontrolled changes to production software.

Expected Security Evidence

• Original vulnerability findings and identifiers.
• Records identifying affected packages, applications, and deployments.
• AI-generated vulnerability analysis and candidate remediation recommendations.
• Proposed dependency changes and source code modifications.
• Dependency compatibility analysis results.
• Build, security scanning, and regression testing results.
• Records of rejected or unsuccessful remediation attempts.
• Pull requests containing validated remediation changes.
• Issues documenting unresolved vulnerabilities.
• Human review and approval records.
• Release and deployment records for corrected software.
• Post-deployment verification showing remediation status.
• Records of AI recommendations and independent validation outcomes.

These artifacts provide evidence that AI-assisted vulnerability remediation follows controlled software development and deployment practices.

The objective is to reduce the time required to investigate and prepare vulnerability corrections without weakening security validation or production change authorization.

Additional Resources

• NIST Secure Software Development Framework
• NIST AI SSDF Profile
• NIST Cybersecurity Framework 2.0
• OSV
• Renovate
• OpenHands
• Ortelius

3.5 - AI Incident Response and Recovery

AI Incident Response and Recovery - Phase 3: Post Deploy

AI-enabled applications introduce security incident scenarios that may differ from traditional application and infrastructure compromises.

Incidents can involve prompt injection, unauthorized agent execution, compromised model artifacts, manipulated retrieval content, sensitive information exposure, or abuse of external tools and services.

These incidents may affect application behavior even when the underlying software binaries remain unchanged.

During Post Deploy, organizations should extend established cybersecurity incident response procedures to include AI-specific threats, affected assets, and recovery requirements.

This activity focuses on detecting and investigating AI security incidents, containing affected systems, preserving evidence, restoring trusted configurations, and incorporating lessons learned into CI/CD security practices.

AI Incident Response and Recovery

Objective: Establish repeatable processes for detecting, investigating, containing, recovering from, and documenting AI-related security incidents while preserving reliable security evidence and preventing recurrence.

The NIST Cybersecurity Framework 2.0 establishes a lifecycle approach to cybersecurity risk management, including detection, response, and recovery.

NIST SP 800-61 Rev. 3 provides incident response recommendations aligned with the NIST Cybersecurity Framework.

The NIST AI Risk Management Framework provides guidance for measuring and managing risks associated with AI systems.

For DevOps teams, incident response should incorporate AI application telemetry, agent execution records, model and configuration inventories, deployment evidence, and software dependency information.

Security teams should be able to identify which AI components were involved, which actions occurred, which resources were affected, and whether the incident involved compromised software, manipulated model behavior, or unauthorized agent activity.

Recovery must restore both trusted software artifacts and approved AI configurations, permissions, models, and data sources.

AI Security Requirements and Open-Source Tooling

The listed tools support incident detection, investigation, containment, and recovery activities. They do not replace an established incident response process or guarantee identification of every AI-specific threat.

AI Incident Response Workflow

A typical AI security incident response process should include:

1. Detection and Triage

Identify suspicious AI behavior, unauthorized agent actions, sensitive information exposure, or infrastructure security events.

Determine the potential impact and initiate the appropriate incident response procedures.

2. Investigation and Impact Assessment

Collect relevant telemetry, agent execution records, model information, configuration versions, and deployment evidence.

Identify affected applications, users, services, and supporting infrastructure.

3. Containment

Restrict compromised services, suspend affected agents, revoke credentials, and prevent additional unauthorized operations.

Containment procedures should preserve evidence where practical.

4. Remediation

Correct identified vulnerabilities, remove malicious configurations or content, replace compromised artifacts, and address unauthorized access paths.

Changes should be validated through established CI/CD workflows.

5. Recovery

Restore approved model versions, prompts, agent configurations, application artifacts, and infrastructure settings.

Verify that recovered services operate within approved security boundaries before resuming normal operations.

6. Lessons Learned

Document root causes, incident impact, remediation outcomes, and opportunities to improve security controls.

Update threat models, security tests, monitoring rules, and deployment policies to reduce the likelihood of recurrence.

Expected Security Evidence

• AI security incident detection records and alerts.
• Incident classification and severity assessments.
• AI application and agent execution logs.
• Model, prompt, and agent configuration version records.
• Affected software component and deployment inventories.
• Security investigation findings and incident timelines.
• Records of contained workloads and suspended agents.
• Credential revocation and access restriction records.
• Remediation changes and associated validation results.
• Recovery procedures and approved rollback records.
• Security and functional testing results following recovery.
• Incident closure and post-incident review reports.
• Updated threat models, monitoring policies, and CI/CD security requirements.

These artifacts demonstrate that AI-related security incidents can be investigated and resolved through structured, repeatable processes.

Security incident findings should inform future development, testing, and deployment activities.

Additional Resources

• NIST Cybersecurity Framework 2.0
• NIST SP 800-61 Rev. 3 - Incident Response
• NIST AI Risk Management Framework
• OWASP Top 10 for LLM Applications
• MITRE ATLAS
• OpenTelemetry
• Falco
• Velociraptor