AI Incident Response and Recovery - Phase 3: Post Deploy
AI-enabled applications introduce security incident scenarios that may differ from traditional application and infrastructure compromises.
Incidents can involve prompt injection, unauthorized agent execution, compromised model artifacts, manipulated retrieval content, sensitive information exposure, or abuse of external tools and services.
These incidents may affect application behavior even when the underlying software binaries remain unchanged.
During Post Deploy, organizations should extend established cybersecurity incident response procedures to include AI-specific threats, affected assets, and recovery requirements.
This activity focuses on detecting and investigating AI security incidents, containing affected systems, preserving evidence, restoring trusted configurations, and incorporating lessons learned into CI/CD security practices.
AI Incident Response and Recovery
Objective: Establish repeatable processes for detecting, investigating, containing, recovering from, and documenting AI-related security incidents while preserving reliable security evidence and preventing recurrence.
The NIST Cybersecurity Framework 2.0 establishes a lifecycle approach to cybersecurity risk management, including detection, response, and recovery.
NIST SP 800-61 Rev. 3 provides incident response recommendations aligned with the NIST Cybersecurity Framework.
The NIST AI Risk Management Framework provides guidance for measuring and managing risks associated with AI systems.
For DevOps teams, incident response should incorporate AI application telemetry, agent execution records, model and configuration inventories, deployment evidence, and software dependency information.
Security teams should be able to identify which AI components were involved, which actions occurred, which resources were affected, and whether the incident involved compromised software, manipulated model behavior, or unauthorized agent activity.
Recovery must restore both trusted software artifacts and approved AI configurations, permissions, models, and data sources.
AI Security Requirements and Open-Source Tooling
| Framework Reference | Security Activity | CI/CD Implementation | Suggested Open-Source Tools |
|---|---|---|---|
| NIST CSF DE.CM, DE.AE | AI security incident detection | Detect suspicious model interactions, unexpected agent execution, unauthorized access, and infrastructure security events. | Falco, Wazuh |
| NIST CSF RS.AN | AI incident investigation | Correlate AI application telemetry, agent tool calls, authentication records, and infrastructure logs. | OpenSearch, OpenTelemetry |
| NIST CSF ID.AM, RS.AN | Affected AI asset identification | Identify affected models, software dependencies, application releases, configurations, and deployed environments. | Ortelius, GUAC |
| NIST CSF RS.AN | Security evidence preservation | Preserve relevant execution logs, security events, deployment records, and model artifact identifiers for investigation. | OpenSearch, Velociraptor |
| NIST CSF RS.MI | Agent and workload containment | Isolate compromised workloads, suspend agent execution, and restrict access to affected services. | Kubernetes, Cilium |
| NIST CSF PR.AA, RS.MI | Credential and permission revocation | Revoke affected identities, rotate credentials, and restrict compromised agent permissions. | Keycloak, Open Policy Agent |
| NIST CSF RC.RP | Trusted artifact recovery | Restore approved application versions, container images, models, prompts, and deployment configurations. | Argo CD, Flux |
| NIST CSF RC.RP | Recovery validation | Execute security and functional validation before restoring affected AI services to production. | Jenkins, Promptfoo |
| NIST CSF RS.MA, RS.AN | Incident coordination and tracking | Maintain incident records, assign investigation responsibilities, and document response actions. | DFIR-IRIS, Gitea |
| NIST CSF ID.IM | Continuous security improvement | Convert incident findings into development security requirements, regression tests, and updated CI/CD policies. | Gitea, Open Policy Agent |
The listed tools support incident detection, investigation, containment, and recovery activities. They do not replace an established incident response process or guarantee identification of every AI-specific threat.
AI Incident Response Workflow
A typical AI security incident response process should include:
1. Detection and Triage
Identify suspicious AI behavior, unauthorized agent actions, sensitive information exposure, or infrastructure security events.
Determine the potential impact and initiate the appropriate incident response procedures.
2. Investigation and Impact Assessment
Collect relevant telemetry, agent execution records, model information, configuration versions, and deployment evidence.
Identify affected applications, users, services, and supporting infrastructure.
3. Containment
Restrict compromised services, suspend affected agents, revoke credentials, and prevent additional unauthorized operations.
Containment procedures should preserve evidence where practical.
4. Remediation
Correct identified vulnerabilities, remove malicious configurations or content, replace compromised artifacts, and address unauthorized access paths.
Changes should be validated through established CI/CD workflows.
5. Recovery
Restore approved model versions, prompts, agent configurations, application artifacts, and infrastructure settings.
Verify that recovered services operate within approved security boundaries before resuming normal operations.
6. Lessons Learned
Document root causes, incident impact, remediation outcomes, and opportunities to improve security controls.
Update threat models, security tests, monitoring rules, and deployment policies to reduce the likelihood of recurrence.
Expected Security Evidence
- AI security incident detection records and alerts.
- Incident classification and severity assessments.
- AI application and agent execution logs.
- Model, prompt, and agent configuration version records.
- Affected software component and deployment inventories.
- Security investigation findings and incident timelines.
- Records of contained workloads and suspended agents.
- Credential revocation and access restriction records.
- Remediation changes and associated validation results.
- Recovery procedures and approved rollback records.
- Security and functional testing results following recovery.
- Incident closure and post-incident review reports.
- Updated threat models, monitoring policies, and CI/CD security requirements.
These artifacts demonstrate that AI-related security incidents can be investigated and resolved through structured, repeatable processes.
Security incident findings should inform future development, testing, and deployment activities.