Continuous AI Vulnerability Management

Identifying vulnerabilities in deployed AI applications, correlating vulnerable dependencies with production environments, and managing AI-specific behavioral security risks.

Continuous AI Vulnerability Management - Phase 3: Post Deploy

AI-enabled applications depend on software frameworks, model-serving infrastructure, libraries, containers, APIs, and supporting services that may become vulnerable after deployment.

A software component that passes security testing during development may later be affected by a newly disclosed Common Vulnerabilities and Exposures (CVE) record.

Traditional pre-deployment scanning cannot detect vulnerabilities that become known after software has already been released.

AI applications also introduce behavioral security risks, including prompt injection, excessive agent permissions, sensitive information disclosure, and manipulation of model outputs. These weaknesses may not have CVE identifiers and cannot always be addressed through traditional software patching.

During Post Deploy, organizations should continuously monitor both newly disclosed software vulnerabilities and AI-specific behavioral security risks.

This activity focuses on maintaining deployed software visibility, identifying affected environments, assessing AI-specific threats, prioritizing remediation, and verifying that corrective actions reach affected production systems.

Continuous AI Vulnerability Management

Objective: Continuously identify, assess, prioritize, and remediate software vulnerabilities and AI-specific behavioral security weaknesses affecting deployed AI applications, dependencies, agents, and supporting infrastructure.

The NIST Secure Software Development Framework (SSDF) establishes practices for identifying vulnerabilities in released software, assessing their impact, and implementing appropriate responses.

The NIST AI SSDF Profile (SP 800-218A) extends secure development guidance to address AI model development and integration.

The NIST Cybersecurity Framework 2.0 provides guidance for identifying technology assets, managing vulnerabilities, assessing risks, and implementing appropriate responses.

The NIST AI Risk Management Framework provides guidance for identifying, measuring, monitoring, and managing risks associated with AI systems.

The OWASP Top 10 for LLM Applications identifies AI-specific security risks, including prompt injection, sensitive information disclosure, improper output handling, and excessive agency.

For DevOps teams, continuous AI security requires two complementary practices:

  1. Software Vulnerability Management: Identify known software vulnerabilities, correlate findings with deployed components, and verify remediation.
  2. AI Behavioral Risk Management: Identify and mitigate AI-specific weaknesses through adversarial testing, runtime monitoring, access controls, and application security improvements.

These practices should share application inventory, ownership, and deployment information while maintaining distinct methods for assessing and remediating security findings.

Software Vulnerability Management - NIST SSDF

Software vulnerability management focuses on identifying, assessing, and remediating known vulnerabilities in deployed software components, including AI frameworks, libraries, containers, model-serving infrastructure, and supporting services.

NIST SSDF practices RV.1 and RV.2 establish processes for identifying and responding to vulnerabilities in released software.

For DevOps teams, vulnerability management must extend beyond creating Software Bills of Materials (SBOMs) and scanning release artifacts.

SBOM information should be associated with application versions and deployment metadata so teams can identify which vulnerable components are actually running, where they are deployed, and who is responsible for remediation.

Continuous monitoring of vulnerability intelligence sources enables organizations to identify vulnerabilities disclosed after deployment and determine their potential impact on production applications.

Remediation should be prioritized based on severity, known exploitation, exposure, affected production systems, and operational risk.

Software Vulnerability Management Requirements and Open-Source Tooling

Framework Reference Security Activity CI/CD Implementation Suggested Open-Source Tools
NIST SSDF PS.3; NIST CSF ID.AM Deployed AI component inventory Maintain records of deployed applications, AI frameworks, model-serving components, software packages, and supporting dependencies. Syft, CycloneDX
NIST SSDF RV.1 Continuous vulnerability intelligence Monitor vulnerability databases for newly disclosed vulnerabilities affecting deployed software components. OSV, OSV-Scanner
NIST SSDF RV.1; NIST CSF ID.RA Deployed vulnerability correlation Associate vulnerability findings with SBOM components, application releases, and deployment records to identify affected environments. Ortelius, GUAC
NIST SSDF RV.1 Container and dependency vulnerability assessment Reassess container images and application dependencies against updated vulnerability intelligence. Trivy, Grype
NIST CSF ID.RA Vulnerability prioritization Prioritize vulnerabilities based on severity, known exploitation, exposure, and affected production assets. OSV, DefectDojo
NIST SSDF PS.3, RV.1 Affected application identification Trace vulnerable components to associated software artifacts, deployments, and application ownership records. Ortelius, GUAC
NIST SSDF RV.2 Remediation tracking Record vulnerability ownership, remediation decisions, affected releases, and outstanding corrective actions. DefectDojo, Gitea
NIST SSDF RV.2 Remediation verification Rescan affected software and confirm that patched components have reached the relevant deployment environments. Trivy, Ortelius

Vulnerability discovery, software supply chain correlation, and deployment inventory are complementary capabilities. An SBOM scanner identifies software components and may detect known vulnerabilities, while deployment correlation is needed to determine which applications and environments are affected.

Software Vulnerability Management - Expected Security Evidence

  • Inventory of deployed AI applications and supporting software components.
  • SBOMs associated with released application versions.
  • Deployment records identifying environments and affected assets.
  • Newly disclosed CVE detection records.
  • Correlations between vulnerable packages and deployed applications.
  • Vulnerability severity, exploitability, and exposure assessments.
  • Documented vulnerability ownership and prioritization decisions.
  • Remediation tracking records and assigned corrective actions.
  • Post-remediation vulnerability scanning results.
  • Evidence confirming deployment of corrected software versions.

These artifacts provide evidence that software vulnerability management continues after deployment and that newly disclosed vulnerabilities can be associated with affected production systems.

Vulnerability findings should initiate appropriate remediation workflows, including the AI-assisted remediation activities described in this phase.

AI Behavioral Risk Management - OWASP and NIST AI RMF

AI-enabled applications introduce behavioral security weaknesses that may not be associated with a known CVE or vulnerable software package. These weaknesses arise from how AI systems process instructions, interact with external content, access tools, and make decisions.

Examples include prompt injection, excessive agency, sensitive information disclosure, unsafe output handling, and manipulation of retrieved information.

Unlike traditional software vulnerabilities, these issues may require changes to application architecture, trust boundaries, authorization policies, model configurations, prompts, or agent permissions rather than dependency upgrades.The OWASP Top 10 for LLM Applications identifies common AI-specific security risks, while the NIST AI Risk Management Framework provides guidance for assessing, monitoring, and managing AI risks throughout the operational lifecycle.

For DevOps teams, AI behavioral risk management should be integrated into continuous security testing, runtime monitoring, application configuration management, and incident response processes. Security findings should be associated with affected AI applications, model versions, agent configurations, and deployed environments whenever possible.

AI Behavioral Risk Management Requirements and Open-Source Tooling

Framework Reference Security Activity CI/CD Implementation Suggested Open-Source Tools
OWASP LLM01; NIST AI RMF MEASURE Prompt injection risk assessment Test and monitor AI applications for malicious instructions delivered through user prompts, retrieved documents, tool outputs, or external content. Promptfoo, Garak
OWASP LLM02; NIST AI RMF MEASURE Sensitive information disclosure Evaluate whether AI application outputs or interactions expose confidential information, credentials, or protected data. Microsoft Presidio, Promptfoo
OWASP LLM05 Unsafe output handling Test whether AI-generated outputs can trigger unauthorized commands, injection attacks, or unsafe downstream operations. Semgrep, OWASP ZAP
OWASP LLM06; NIST AI RMF MANAGE Excessive agency and agent permissions Evaluate agent permissions, tool access, and autonomous actions to prevent operations beyond approved authorization boundaries. Open Policy Agent, Promptfoo
OWASP LLM08; NIST AI RMF MEASURE Retrieval and vector security Assess unauthorized retrieval, malicious document injection, and manipulation of external information used by AI applications. Promptfoo, Garak
NIST AI RMF MEASURE, MANAGE Continuous AI security evaluation Execute repeatable adversarial evaluations and reassess AI application security following changes to models, prompts, tools, or external dependencies. Inspect AI, Promptfoo
NIST AI RMF MANAGE; NIST CSF DE.CM AI runtime security monitoring Collect application telemetry, monitor abnormal agent behavior, and correlate AI security events with supporting infrastructure activity. OpenTelemetry, Falco
NIST AI RMF MANAGE AI behavioral risk tracking Record identified weaknesses, affected AI components, mitigation decisions, and unresolved security risks. DefectDojo, Gitea

OWASP identifies AI application security risk categories, while NIST AI RMF provides processes for managing AI risks. The suggested controls and tools are implementation examples rather than mandatory requirements specified by these frameworks.

AI Behavioral Risk Management - Expected Security Evidence

  • AI application threat models and behavioral risk assessments.
  • Prompt injection testing and evaluation results.
  • Sensitive information disclosure assessments.
  • Agent permission and authorization validation records.
  • Security testing results for retrieval and external tool integrations.
  • AI model and application evaluation reports.
  • Runtime monitoring and security event records.
  • Documented AI behavioral weaknesses and affected applications.
  • AI security risk ownership and prioritization decisions.
  • Mitigation plans and security configuration changes.
  • Post-mitigation security testing and validation results.
  • Records of accepted, unresolved, or escalated AI security risks.

These artifacts provide evidence that AI-specific behavioral security risks are being assessed and managed throughout the operational lifecycle. Because AI behavior may vary with model versions, prompts, retrieved content, and external integrations, security evaluations should be repeated when significant changes occur.

Coordinating Software Vulnerabilities and AI Behavioral Risks

Software vulnerability management and AI behavioral risk management should operate as complementary security processes. A deployed AI application may contain vulnerable software dependencies while also exhibiting unsafe behavior caused by excessive agent permissions, prompt injection, or insecure application design.

Both categories of findings should be linked to the affected applications and deployment environments, but their detection and remediation requirements may differ.

Security Consideration Software Vulnerability Management AI Behavioral Risk Management
Primary Guidance NIST SSDF OWASP LLM Top 10, NIST AI RMF
Security Findings CVEs, vulnerable dependencies, software weaknesses Prompt injection, excessive agency, data exposure, unsafe model interactions
Detection Methods Vulnerability intelligence, SCA, SBOM analysis, artifact scanning Adversarial testing, behavioral evaluation, runtime monitoring
Affected Assets Software packages, containers, frameworks, infrastructure Models, prompts, agents, retrieval systems, application configurations
Typical Corrective Actions Dependency upgrades, patches, configuration updates Access control changes, application fixes, prompt changes, additional security boundaries
Validation Rescanning, regression testing, deployment verification Security reevaluation, adversarial testing, policy validation
Evidence CVE records, SBOMs, patch records, deployment history Evaluation results, agent logs, policy changes, behavioral risk assessments

For DevOps and platform engineering teams, maintaining a connection between these security processes improves visibility into the complete security posture of deployed AI applications.

A common application inventory, ownership model, and deployment record can help correlate findings while preserving the distinct evaluation and remediation workflows required for each risk category.

Additional Resources