There is a threshold that AI systems cross when they stop suggesting things to humans and start doing things in the world. On one side of the threshold, the AI is a tool. It produces outputs; humans decide what to do with them. Traditional AI governance was built around this model. Human oversight sits between the AI’s output and any consequence in the world. If the AI is wrong, the human catches it. If the AI is biased, the human corrects it. The AI’s failure mode is bounded by the human review that sits downstream.
On the other side of the threshold, the AI takes actions. It sends emails, executes trades, updates records, calls APIs, purchases things, generates code that gets deployed, files reports that get submitted. The human is not always downstream. Sometimes the human is upstream, defining goals for the agent. Sometimes the human is nowhere near the specific action at all.
Crossing this threshold changes governance. The controls that worked on the “tool” side do not fully translate. New failure modes appear. Traditional security assumptions bend. And the pace at which enterprises are crossing the threshold has accelerated dramatically through 2025 and 2026.
This article is about governing AI systems that take action. It covers what agentic AI is (with more precision than the marketing terminology usually offers), the specific risk categories that agentic systems introduce, and the governance approaches that are emerging as the standard response. Batch 1 article 3 flagged agentic AI as a category we would come back to; this is where we come back to it.
What actually counts as agentic
The word “agent” is used loosely across the industry, so let me be more precise.
An agentic AI system has some combination of the following properties:
Goal-directed autonomy. The system pursues a defined objective across multiple steps, choosing its own sub-actions along the way. This is more than a workflow; it is genuine multi-step reasoning about how to achieve a goal.
Tool use. The system can invoke external tools: APIs, databases, applications, code execution environments, other AI models. Tool use is what lets the system act rather than merely output text.
Persistent state or memory. The system maintains information across interactions or steps. This lets it work on longer-horizon tasks and interact more contextually with its environment.
Environmental interaction. The system reads from and writes to external state: files, databases, cloud services, physical devices. Its actions affect the world in ways that persist.
Delegation and orchestration. In multi-agent systems, agents may delegate sub-tasks to other agents, coordinate action, or negotiate. This adds a further layer of complexity.
Not every system labelled “agent” has all of these properties. A chatbot that can call a single API is at the low end; a fully autonomous system that plans multi-day projects and coordinates with sub-agents is at the high end. The governance implications scale with where a system sits on this spectrum.
The Singapore IMDA, in its January 2026 Model AI Governance Framework for Agentic AI, described agentic systems as those where “AI models are used, but with a level of autonomy and initiative, to plan and execute actions on behalf of the user.” The Framework identifies specific risks that agentic systems introduce beyond generative AI.
Why traditional controls do not fully translate
Three specific ways that the “AI as tool” governance model breaks down when action is added.
The blast radius expands. For a tool AI, the failure mode is a wrong output. The human sees it, ignores it, and the world is unchanged. For an agent, the failure mode is a wrong action. By the time the human sees it, the email has been sent, the trade has been executed, the record has been updated. Undoing is often expensive, sometimes impossible.
Attack surfaces multiply. A tool AI has one input surface: the user’s query. An agent has many: the user’s query, retrieved documents, tool outputs, memory contents, other agents’ messages. Each of these is an attack surface. Prompt injection through retrieved content or tool outputs is a documented enterprise concern; the OWASP Top 10 for Agentic Applications 2026 identifies memory poisoning, tool misuse, and privilege compromise as top-tier risks.
Authorisation becomes central. For a tool AI, authorisation is straightforward: the user is authorised to see the output. For an agent, authorisation is complex: the agent needs credentials to invoke tools, those credentials confer capability, and the agent’s decisions determine what those credentials get used for. Traditional identity and access management was not designed for AI actors whose decisions determine what actions get taken with delegated credentials.
The result is that agentic AI governance is not just AI governance with a bit more paperwork. It is a distinct discipline that draws on identity and access management, security architecture, transaction monitoring, and traditional AI governance, integrated in a way that most enterprises are only beginning to figure out.
The risk landscape for agents
The OWASP Top 10 for Agentic Applications 2026, released in December 2025, is the industry’s most comprehensive current catalogue of agent-specific risks. It uses the identifier ASI (Agent Security Initiative) with numbers 01 through 10. Without reproducing the full list, the categories cluster around:
Memory and context risks. Persistent memory can be poisoned. Context injected from documents or tool outputs can manipulate agent behaviour. These are variants of prompt injection with longer time horizons.
Tool and privilege risks. Agents with tool access can misuse tools, escalate privileges beyond intended scope, or be manipulated into invoking tools an attacker wants invoked.
Reasoning and goal risks. Agents can be manipulated into pursuing different goals than the ones they were assigned. They can hallucinate about their own progress, leading to cascading failures. They can be manipulated into intent-breaking behaviour.
Communication and coordination risks. In multi-agent systems, communication between agents introduces vulnerabilities. Insecure inter-agent communication can be intercepted or spoofed. Cascading failures can propagate through agent networks.
Trust and behavioural risks. Users can over-trust agents, agents can exploit user trust, and agents can develop rogue behaviour where their actions diverge from intended purpose.
These risk categories overlap with, and extend, the general AI risks we covered in article 3. Some (memory poisoning, tool misuse) are agent-specific. Others (cascading hallucination) are generative AI risks amplified by the agentic context.
Figure 1: The agent risk map
Figure 1 organises agent-specific risks by where they enter and where they manifest. Input-side risks (memory poisoning, context injection) enter through the agent’s inputs and manifest in altered behaviour. Reasoning-side risks (goal manipulation, cascading hallucination) originate in the agent’s decision process and manifest in incorrect actions. Action-side risks (tool misuse, privilege compromise) originate in the agent’s tool invocations and manifest as real-world consequences. Coordination risks (inter-agent communication vulnerabilities, cascading failures) span multiple agents and manifest at the system level. Each risk category requires different controls at different points.
Governance responses that are emerging
Several governance patterns for agentic AI have started to consolidate through 2025 and 2026.
Least-privilege tool access. Agents should only have access to the tools they need to accomplish their assigned tasks. Blanket access to email, file systems, and enterprise applications is not defensible for most use cases. Tool access should be scoped, auditable, and revocable.
Action authorisation checkpoints. For actions with material consequences (spending money, sending external communications, modifying production systems), an explicit authorisation step should sit between the agent’s decision and the execution. This may be human authorisation for high-stakes actions or programmatic policy checks for lower-stakes ones.
Action budgets and rate limits. Agents should operate within defined action budgets: total number of actions per session, total resource consumption, spending limits. This bounds the blast radius when things go wrong.
Behavioural monitoring. Beyond input/output monitoring, agents need behavioural monitoring: sequences of actions, tool usage patterns, memory contents, deviations from expected behaviour envelopes. Anomaly detection at the behaviour level catches issues that per-step monitoring misses.
Rollback and unwind capabilities. Where possible, agent actions should be reversible. Where they cannot be, agents should be prevented from taking them without explicit authorisation. This is a design constraint that affects what tools agents are given access to.
Multi-agent architecture governance. Where multiple agents interact, governance must consider the system as a whole. Communication channels between agents need security controls. Agent identity must be verifiable. Coordination must be observable.
Human oversight redesigned. Traditional human oversight (review the AI’s output before it is acted on) does not translate cleanly to agentic systems. New patterns are emerging: policy-based control (the human sets policies that the agent must operate within), episodic review (the human reviews sessions rather than individual actions), and structural constraints (the environment enforces limits that cannot be exceeded regardless of what the agent decides).
Emergency stop mechanisms. Every agentic system should have a stop mechanism: a way to halt an agent’s operation quickly when something goes wrong. This is analogous to circuit breakers in trading systems.
The Singapore IMDA framework
The IMDA’s January 2026 Model AI Governance Framework for Agentic AI, positioned as the world’s first framework specifically for agents, is worth understanding. It identifies four broad governance considerations:
Purpose and boundaries. Clear definition of what the agent is designed to do, what it is authorised to do, and what is out of bounds. Documented boundaries that the agent cannot cross regardless of instruction.
Human control. Meaningful human control over agent behaviour, appropriate to the risk. This includes ability to set goals, monitor progress, and intervene where necessary.
Assurance. Ongoing evaluation of agent behaviour against defined criteria, with attention to emergent behaviours that were not anticipated at design time.
Transparency. Users and affected individuals should understand when they are interacting with an agent versus a human, and what the agent is authorised to do.
Under all four considerations, the Framework emphasises risk proportionality. Higher-risk agentic use cases attract more stringent expectations. The Framework is voluntary but is being cited as reference practice by financial regulators and by other jurisdictions beginning to consider agentic AI governance.
The NIST work in progress
NIST’s AI Agent Standards Initiative, launched February 2026 through the Center for AI Standards and Innovation, is producing voluntary guidelines specifically for AI agents. The initiative covers three areas: identity and authorisation (how agents are identified and what they are authorised to do), security and risk management (how agents are protected from adversaries and how their risks are managed), and monitoring and logging (how agent behaviour is observed and recorded).
The AI Agent Interoperability Profile is planned for Q4 2026. The Cloud Security Alliance has published a proposed Agentic Profile for the AI RMF in April 2026, which extends the four RMF functions with agent-specific subcategories. This is not official NIST output but indicates the direction that formal guidance is likely to take.
For enterprises implementing agentic systems now, the pragmatic approach is to build against emerging best practice (the OWASP Top 10, the IMDA framework, the CSA proposals) with the expectation that formal standards will follow and can be integrated later.
The specific challenge of large-scale multi-agent systems
Two additional considerations for enterprises considering multi-agent architectures.
Cascading failure risk. In multi-agent systems, a failure in one agent can propagate through the system as other agents consume its outputs. Traditional software fault isolation techniques (circuit breakers, bulkheads, retries with backoff) apply but may not be sufficient. Multi-agent systems can produce failure modes that no individual agent’s design contemplated.
Emergent behaviour. Multi-agent systems can produce collective behaviours that emerge from their interactions, not from any single agent’s design. Some of these emergent behaviours are useful (that is often the point of the architecture); some are undesirable and hard to anticipate. Governance must include mechanisms for detecting and responding to emergent behaviour, and this is genuinely difficult research territory.
For most enterprises in 2026, single-agent systems represent the pragmatic frontier of production deployment. Multi-agent systems are being piloted but not yet widely deployed at scale in high-risk contexts. The governance framework needs to be ready for both.
Contract and vendor considerations
Agentic AI introduces vendor considerations beyond those for traditional AI.
Model behaviour under updates. Foundation model providers update their models. An agent’s behaviour depends on the underlying model, so provider updates can change agent behaviour without any deployer change. Contracts should address this: notification of material updates, version pinning options, testing time before mandatory updates.
Tool availability. Agent capability depends on tool availability. When tools change or become unavailable, agent behaviour is affected. Contracts and monitoring should address this.
Data flow. Agents may pass data to multiple external services in the course of their operation. Data flow governance must address the specific services and the specific data.
Liability allocation. When an agent takes a wrong action, who is responsible? The provider of the model? The provider of the tool that was misused? The deployer who configured the agent? Contract terms are beginning to grapple with this; most current terms are not clear enough.
Article 16 covers third-party AI risk in general; the agentic case sharpens the specific questions but does not change their fundamental structure.
Figure 2: The agentic AI control stack
Figure 2 shows the control stack for agentic AI systems. At the base are design-time controls: purpose definition, tool scoping, authorisation architecture. Above them are development-time controls: adversarial testing, behavioural evaluation, red-teaming for agentic-specific risks. Above them are deployment-time controls: policy enforcement, action monitoring, rate limits, budgets. At the top are runtime controls: real-time behavioural monitoring, human escalation, emergency stops. Each layer supports the layer above; gaps in any layer create risk that other layers cannot fully compensate for.
Where this leaves the practitioner
Three practical moves for organisations moving into agentic AI.
Tier agentic use cases separately. Do not assume that a use case that is medium-risk as a tool AI is medium-risk as an agent. The action layer changes the risk profile. Reassess.
Invest in observability early. The hardest thing about agentic AI in production is understanding what it is actually doing. Investment in behavioural monitoring, action logging, and structured observability pays back the first time an incident needs to be investigated.
Design for constrained autonomy. The strongest pattern I see working in production is agents with meaningful autonomy within tightly bounded scope. Broad autonomy in high-stakes contexts is not yet safely achievable. Tight scoping is a design choice, not a limitation to be worked around.
The next article covers third-party AI risk and shadow AI, which are the two remaining risk categories where AI governance is still catching up to reality. Article 17 then closes with incident response, red-teaming, and the practical governance checklist.