This is the closing article of the track, and I want it to earn its place as the operational closer. Everything we have covered so far (board-level attention, the risk-tiered philosophy, the specific risk categories, the jurisdictional regimes, the frameworks, the model risk management discipline, the documentation practices, the responsible AI practices, the agentic AI considerations, the third-party and shadow AI landscape) all leads here: to what you actually do on Monday morning.
Three disciplines matter for that Monday morning. The first is incident response: when something goes wrong with an AI system, what happens, how quickly, and by whom. The second is red-teaming: how you find the things that could go wrong before they do. The third is audit readiness: how you demonstrate to regulators, boards, internal auditors, and customers that your governance program is real rather than aspirational.
The article closes with a practical governance checklist. It is tiered to the philosophy from article 2, mapped to the regulatory expectations from batch 1, and grounded in the frameworks from batch 2. It is designed to be a real starting point for a real program, not a theoretical exercise.
AI incident response: what makes it different
Incident response is not new. Every large organisation has an incident response capability of some kind, typically built around cyber incidents. What makes AI incident response distinctive is what counts as an incident and how you find out about it.
For traditional cyber incidents, the triggers are relatively well-defined: unusual network activity, data exfiltration alerts, malware detections, user reports. The definition of incident is standardised across the industry, and the response playbooks are mature.
For AI incidents, the triggers are more varied and less clear-cut. An AI system producing biased outputs is an incident, but the detection requires ongoing fairness monitoring. A hallucination that reaches a customer is an incident, but not every hallucination is caught before it reaches a customer. A prompt injection that alters agent behaviour is an incident, but distinguishing it from normal variability is non-trivial. A vendor changing their model’s behaviour is not obviously an incident until its consequences show up in downstream systems.
The result is that AI incident response needs to sit at the intersection of cyber incident response, model risk management, and operational risk management. It draws on all three disciplines and belongs cleanly to none of them.
Under the EU AI Act, providers of high-risk systems must report serious incidents to national competent authorities within specific timeframes (Article 73). The definition of serious incident is broad, covering harm to health, fundamental rights, and critical infrastructure. GPAI providers with systemic risk must also report incidents to the AI Office (Article 55).
Under APRA CPS 234, notifiable information security incidents include AI-specific security events. Under CPS 230, operational incidents include AI-related failures materially affecting operations. Both frameworks require timely notification.
Under MAS Guidelines and Notice 655 (Cyber Hygiene), incident reporting obligations apply to AI-related events that meet the notification thresholds. Under Notice 644 (Technology Risk Management), systemic issues require reporting.
The result is a reporting landscape where an AI incident may need to be reported to multiple regulators simultaneously, on different timelines, with different content requirements. Incident response processes need to be designed for this.
What an AI incident response capability looks like
Six elements make an AI incident response capability effective.
Definition and triggers. What counts as an AI incident in your organisation is documented. Triggers cover model performance degradation beyond defined thresholds, fairness metric excursions, hallucination patterns, prompt injection detection, unauthorised model use, vendor-side model changes with material behavioural impact, and any AI-related event that would ordinarily trigger existing incident response processes (security, privacy, operational).
Detection. How incidents are detected in the first place. This includes monitoring outputs (from article 12), user reports, third-party notifications (from vendors, regulators, or researchers), and periodic assessment cycles. Detection sits at the intersection of technical monitoring and organisational culture: employees and users must know how to report AI concerns.
Triage and severity classification. When an incident is detected, it is triaged for severity. Severity determines response urgency, escalation level, and stakeholder notification. The severity classification typically aligns with existing incident response classifications, extended for AI-specific factors.
Response playbook. Documented actions for each incident type: initial containment, investigation, decision-making authority, communication protocol, remediation, and closure. Playbooks should exist for the specific incident types you expect (bias incidents, hallucination incidents, security incidents on AI systems, agent action incidents, vendor incidents). Generic playbooks are not sufficient for AI-specific incident types.
Communication and reporting. Internal escalation to appropriate authorities (CRO, CIO, general counsel, board where applicable). External notification to regulators where required. Communication to affected individuals where appropriate. Communication to customers, partners, or other stakeholders based on materiality.
Post-incident review. Every material incident gets a post-incident review. Root causes are identified. Systemic issues are surfaced. Improvements are captured in updated playbooks, monitoring, or system design. Post-incident review is where incident response transitions from reactive to learning.
The specific case of vendor-side incidents
One category of AI incident deserves specific attention because it catches organisations off guard: vendor-side incidents.
When your foundation model provider has an incident (model behaviour issue, security incident, terms of service change, service disruption), it may cascade to your deployments in ways that were not obvious in advance. A change in the provider’s content policy may cause your customer-facing chatbot to refuse queries it previously handled. A model update may change the fairness characteristics of your credit decision system. A security incident at the provider may require you to reassess data flows, at speed.
Your incident response capability needs to include:
Vendor incident notification channels. Contract obligations for the vendor to notify you of specified events, with specified content, within specified timeframes.
Vendor incident monitoring. Where contractual notification is inadequate or not fully reliable, active monitoring of vendor announcements, model version changes, and public incident disclosures.
Downstream impact assessment. When a vendor incident occurs, your response includes assessing which of your systems are affected, in what way, and with what materiality.
Vendor coordination protocols. Direct communication paths with vendors for material events. For foundation model providers with enterprise agreements, this may be a named contact. For providers with only standard terms, it may be their public incident channels.
For Australian regulated entities, CPS 230 addresses this specifically for material service providers. For EU AI Act purposes, deployers must have a way to detect and respond to material provider changes.
Figure 1: The AI incident response lifecycle
Figure 1 shows the AI incident response lifecycle from detection through post-incident learning. Detection sits at the top, with multiple input channels. Triage and severity classification determine the response path. Containment, investigation, remediation, and communication proceed in parallel where warranted. Regulatory notification runs on defined timeframes independent of internal remediation status. Post-incident review closes the cycle and feeds improvements back into detection, playbooks, and system design. The lifecycle is continuous; every incident produces learning that reduces the likelihood or severity of the next one.
Red-teaming for AI systems
Red-teaming for AI is the discipline of adversarial testing: systematically attempting to make the system fail in the ways it should not fail, before an adversary or an unlucky user does it for you.
Red-teaming for traditional software is well-established. Penetration testing, vulnerability assessment, and adversarial code review are mature disciplines. Red-teaming for AI extends these into AI-specific failure modes.
Effective AI red-teaming addresses several categories of adversarial testing.
Model behavioural red-teaming. Attempting to make the model produce outputs it should not produce: harmful content, biased outputs, confidential information disclosure, incorrect but authoritative-sounding outputs. This is closest to the traditional model evaluation discipline but with an adversarial framing.
Prompt injection red-teaming. Attempting to manipulate model behaviour through crafted inputs, indirect prompt injection through retrieved content, and social engineering of instruction-following behaviour.
Agentic behaviour red-teaming. For agentic systems, attempting to manipulate the agent into taking actions outside its intended scope, misusing tools, or breaking authorisation constraints.
System-level red-teaming. Testing the integration of the AI system with other systems, including data flows, authentication, authorisation, and observability. Failures at the integration layer are often more consequential than failures at the model layer.
Value chain red-teaming. Attempting to compromise the AI system through third-party components, dependencies, or upstream providers.
The MAS Guidelines reference red-teaming as an expected practice for high-materiality AI systems. The EU AI Act GPAI Code of Practice references red-teaming as expected for models with systemic risk. NIST’s ARIA program provides evaluation infrastructure that includes red-team dimensions. Singapore’s AI TAP accredits third-party organisations to perform AI testing including red-teaming.
For enterprises, the pragmatic questions are: what to red-team, how often, by whom, and how to close the findings.
Practical red-teaming for enterprises
What to red-team. High-tier AI systems should be red-teamed before deployment and at defined intervals thereafter. Medium-tier systems should be red-teamed at deployment. Low-tier systems typically do not warrant dedicated red-teaming beyond standard evaluation.
How often. Initial red-teaming before deployment. Regular red-teaming at intervals that reflect risk (quarterly for the highest-tier systems, less frequently for lower tiers). Event-driven red-teaming after material changes: model updates, use case expansion, integration changes.
By whom. Internal red-teams have the advantage of context and continuity. External red-teams (including accredited providers under programs like AI TAP) have the advantage of independence and fresh perspective. The strongest programs use both. For regulated entities, external red-teaming provides evidence of independent assurance.
How to close findings. Red-team findings feed into a documented remediation process. Findings are classified by severity, assigned to owners, tracked to closure, and validated after remediation. The finding register is a governance artefact that regulators may request.
The Cloud Security Alliance’s proposed Agentic Profile references red-teaming under MEASURE extensions. The OWASP Top 10 for Agentic Applications includes red-team-informed threat modelling. Both are useful references for internal red-team methodology.
Audit readiness
Audit readiness is the discipline of being able to demonstrate, to a competent independent reviewer, that your AI governance program is operating as designed. This is what regulators check. This is what internal auditors verify. This is what customers ask about in procurement processes.
Three categories of audit are worth considering.
Internal audit. Your organisation’s internal audit function, or an equivalent second-line review, verifies that governance is operating. Internal audit findings feed into management reporting and board oversight. Under APRA CPS 220 and CPS 510, and under equivalent frameworks elsewhere, internal audit’s coverage of AI is an expected practice.
External audit for certification. ISO/IEC 42001 certification involves external audit. SOC 2 reports involve external audit. These audits produce specific outputs (certificates, reports) that have external commercial and regulatory value.
Regulatory examination. Direct supervisory review by the applicable regulator. In Australia, APRA and ASIC conduct such reviews. In Singapore, MAS conducts them. In the EU, national competent authorities and the AI Office conduct them under the AI Act.
All three audit types look for similar things: policy exists and reflects current practice, processes are documented and being followed, evidence exists that decisions were made in accordance with policy, monitoring is in place and being acted upon, incidents are being detected and responded to, and improvement is happening in response to identified issues.
What audit-ready looks like
An audit-ready AI governance program has certain characteristics that are visible from the first few hours of any audit.
The inventory is current. Every AI system in operation is in the inventory. The inventory is dated within an appropriate timeframe. Entries are complete for the fields the auditor cares about.
Classifications are consistent. Every AI system has a documented risk tier, matched to the tiering methodology, with supporting rationale.
Documentation is live. Model cards, system cards, risk assessments, and impact assessments are current. Version control shows updates over time.
Evidence is available. For each material decision (system approval, risk acceptance, incident closure, model update), the evidence trail exists and is retrievable.
Monitoring outputs are being consumed. Dashboards exist, are looked at, and produce actions when they show issues. Alerts fire and are handled.
Incidents are being learned from. Post-incident reviews exist. Actions from those reviews have been implemented and can be shown.
Reporting flows to the right levels. Board reporting exists and covers material AI governance topics. Executive committees receive regular reports. Second-line risk sees what it needs to see.
People know what they are doing. Interviews with staff at various levels reveal understanding of the governance framework and their role in it. This is often what auditors weigh most heavily, because policies and documents can be written; understanding cannot be faked easily.
Figure 2: The audit-ready control map
Figure 2 shows the control map that supports audit readiness across the three audit types. Governance controls (policy, structure, sign-off) support all three. Documentation controls (inventory, cards, records) support all three. Operational controls (monitoring, incident response, red-teaming) support all three. Assurance controls (internal audit, external assurance, certification) support external audit and regulatory examination specifically. The map is not about doing more; it is about ensuring the controls that exist are connected, evidenced, and durable enough to withstand scrutiny.
The AI governance checklist
Here is the practical checklist. It is tiered to the philosophy from article 2, mapped to the regulatory expectations from batch 1, and grounded in the frameworks from batch 2. Not every item applies to every organisation; apply proportionately to your scale, complexity, and risk profile.
Foundation
- AI policy exists, is signed off at appropriate executive level, and is current within twelve months
- AI risk appetite is documented and consistent with the broader enterprise risk appetite
- Board or equivalent governing body receives regular reporting on AI governance
- AI-specific accountability roles are defined: sponsor, owner per system, second-line risk, internal audit
- AI literacy programme exists for board, executives, risk owners, and technical teams
- Legal, compliance, privacy, security, and procurement functions are engaged in the AI governance framework
Inventory and classification
- AI inventory covers built, purchased, embedded, and shadow AI to the extent identifiable
- Every AI system has a documented tier, applied using a consistent methodology
- Prohibited use cases are documented and communicated
- AI system inventory is reviewed and updated at defined intervals
Risk assessment and impact
- Risk assessment methodology exists, appropriate to organisation size and sector
- Each high-tier and medium-tier system has a documented risk assessment
- Impact assessment (under ISO/IEC 42005 or equivalent) exists for systems affecting individuals materially
- Fundamental Rights Impact Assessment (or equivalent) exists for EU AI Act high-risk deployer scenarios
- Fairness definition per material use case is documented, with justification
Data governance for AI
- Training data provenance is documented for internally-built models
- Data appropriateness is assessed against the use case
- Data bias assessment is performed and documented
- Retrieval corpora for RAG systems are governed with appropriate discipline
- Employee use policies address data leakage through consumer AI tools
Model governance
- Model risk management framework applies to AI models, with tier-appropriate intensity
- Independent validation is performed for high-tier models
- Model cards or equivalent exist for all material models
- Continuous monitoring is in place for production models
- Change management addresses model updates, provider changes, and prompt modifications
Responsible AI operations
- Explanation architecture is defined per use case (model-level, case-level, process-level)
- Human oversight design counters automation bias for high-risk decisions
- Case-level explanations are available where required by law
- Users and affected individuals are informed of AI use where required
- Appeal or human review paths exist for adverse decisions
Third-party and vendor risk
- Material AI vendors are identified and assessed
- Concentration risk is mapped and documented
- Contracts with material AI vendors address AI-specific concerns
- Exit strategies exist for material dependencies and are periodically tested
- For Australian regulated entities, CPS 230 vendor contract obligations are met
Agentic AI
- Agentic use cases are tiered separately from tool-use cases
- Tool access follows least-privilege
- Action authorisation checkpoints exist for material actions
- Action budgets and rate limits are in place
- Emergency stop mechanisms exist and have been tested
Incident response and red-teaming
- AI incident definition and triggers are documented
- Response playbook exists for AI-specific incident types
- Regulatory notification obligations are mapped and processes exist to meet them
- Red-teaming is performed for high-tier systems before deployment and periodically thereafter
- Post-incident review is documented and feeds improvement
Assurance and audit
- Internal audit coverage of AI governance is scheduled and executed
- External assurance mechanisms are in place appropriate to organisational needs (SOC 2, ISO 42001, sector-specific attestations)
- Evidence retention supports audit and regulatory requests
- Findings from any audit type are tracked to closure
Continuous improvement
- The AI governance framework is reviewed against regulatory developments at defined intervals
- Framework updates are documented and communicated
- Metrics on framework effectiveness are collected and reviewed
- Lessons from incidents, red-team findings, and audit findings feed into framework updates
Where this track ends
Seventeen articles. Two batches. A working framework for AI governance built from board-level context, through the risk landscape and jurisdictional map, into the frameworks and controls that make the discipline operational.
Nothing in this track will substitute for the specific work you have to do in your specific organisation. But if you have followed the track, you have the shape of the problem and the shape of the response. The specific frameworks (NIST, ISO 42001), the specific regulatory regimes (EU, US, Australia, APAC), the specific disciplines (model risk management, documentation, responsible AI practices, agentic AI, third-party and shadow AI, incident response and audit readiness) all connect into a coherent whole.
What is next is doing the work. If you can complete the checklist above, at the intensity appropriate to your organisation, you will have an AI governance program that regulators recognise as credible, that boards can defend, and that actually reduces the operational risk of AI in your business. That is what the whole track has been aiming at.
Good luck with the work.