AI Agents & Autonomy  ·  Research Theme  ·  2026

AI Agents and Autonomy: Governing Systems That Act, Not Just Advise

What changes when AI moves from answering questions to planning multi-step actions and executing them, and the controls boards and risk teams need before granting agents real-world permissions.

Read Article
Executive Summary
An AI system that can only advise fails safely. An AI system that can act does not.

Agentic AI, systems that can plan a sequence of steps and execute them with limited or no human confirmation at each step, moved from research demonstration to genuine enterprise deployment faster than most governance frameworks anticipated. By 2026, agents are being used to carry out multi-step tasks across tools and systems, and in a growing number of cases, to take actions that touch customers, money or operational systems directly.

This is a meaningfully different risk profile from the generative AI capability most governance frameworks were built around. A chatbot that produces a poor answer creates a bad output someone has to notice and act on. An agent that takes a wrong action has already changed something in the world, a message sent, a transaction initiated, a system updated, before anyone necessarily has the chance to intervene.

Traditional access control and change management models, built around individual users with stable permissions and predictable behaviour, were not designed for a system that decides its own sequence of actions in response to a goal, and that may behave differently given slightly different inputs even when nothing about its permissions has changed.

This report examines what genuinely changes when AI systems act rather than only advise, why conventional controls break down for agents, and the practical controls, permission boundaries, action logging, escalation paths and adversarial testing, that organisations need before granting agents access to real systems and real consequences.

It also sets out the questions boards and risk committees should be asking right now, before agentic AI capability outpaces the organisation's ability to govern what it is actually authorised to do.

01 · What Actually Changes When AI Plans and Acts
A system that recommends an action can be wrong without consequence until a person acts on it. A system that takes the action itself removes that final checkpoint.

Most enterprise AI deployed to date sits behind a meaningful checkpoint: a person reads a recommendation, a drafted response or an analysis, and decides whether to act on it. Even when that review is imperfect, it exists as a structural barrier between an AI output and a real-world consequence.

Agentic AI is defined by its ability to remove that checkpoint, in whole or in part. An agent given a goal can decide the steps needed to achieve it, call the tools and systems required, and carry those steps out, potentially across several systems and several actions, before a human is necessarily in a position to review what happened.

This changes the nature of an error. A generative system that produces an inaccurate answer has created a piece of content that still needs someone to act on it to cause harm. An agent that takes an incorrect action, sending a communication, moving funds, modifying a record, updating a customer account, has already caused the consequence directly. There may be no intermediate step left to catch it.

It also changes the nature of intent. An agent pursuing a defined goal may find an unexpected, technically valid route to achieving it that nobody anticipated when the goal was set, exploiting a permission, a tool combination or a sequence of steps that was never explicitly considered by the people who designed the system.

None of this means agentic AI should not be deployed. It means the risk conversation has to move from is the output accurate to what is this system actually authorised to do, on its own, before anyone reviews it, and that is a fundamentally different governance question.

02 · Why Traditional Access Control Breaks Down for Agents
Access control was built around predictable users with stable roles. Agents introduce a principal that behaves differently depending on the goal it has been given.

Conventional access control assumes a relatively stable relationship between an identity and the permissions it holds: a person has a role, the role has defined access, and that access changes infrequently and through a known process. Agents disrupt this model in several ways at once.

First, an agent's behaviour is not fully predictable in advance. Two runs with slightly different inputs, or a slightly different phrasing of the same goal, can produce different sequences of actions, even where the underlying permissions are identical. Access control designed around predictable behaviour is being asked to govern something that is deliberately flexible in how it achieves an objective.

Second, agents frequently need to combine access across multiple systems to complete a task, read from one system, write to another, trigger a third. Each individual permission might look reasonable in isolation. The combination can create a capability that nobody explicitly authorised, simply by chaining together access that was granted for separate, narrower purposes.

Third, and most structurally, an agent is often acting nominally on behalf of a human or a business process, which raises a genuine identity question: whose permissions should the agent actually be exercising, its own, a service account's, or a delegated subset of a specific person's access, and how is that choice reflected, audited and constrained in practice.

Addressing this requires access control models built specifically for agents rather than adapted from user-based models: permissions scoped tightly to the task at hand, explicit limits on which systems and actions can be combined, and a clear, auditable answer to whose authority the agent is actually operating under at any given moment.

03 · Permission Boundaries: Designing What an Agent Is Allowed to Do
The most important governance decision for an agent is made before it ever runs: what it is permitted to touch, and what it is never permitted to touch regardless of the goal it is pursuing.

Effective agent governance starts with permission boundaries defined explicitly and narrowly, rather than inherited broadly from an existing service account or a convenient but overly generous role. An agent should generally be granted the minimum access required for its specific task, not the access that happens to be available and technically sufficient.

This includes explicit boundaries on irreversible or high-consequence actions: which actions, if any, the agent can take entirely on its own, which require a human confirmation step before execution, and which should never be within the agent's authority regardless of how confident its reasoning appears to be.

It also means designing boundaries around combinations of actions, not only individual ones. An agent permitted to read customer data and separately permitted to send external communications may, in combination, be capable of an action that neither permission alone was intended to enable. Boundary design has to consider what the agent can do with its full permission set together, not each permission in isolation.

These boundaries need to be treated as a live control, reviewed as the agent's role or the systems it touches change, not configured once at deployment and left unexamined. An agent whose task scope quietly expands over time, a common and often well-intentioned pattern, can end up operating with a permission footprint nobody explicitly approved.

The organisations managing this well tend to apply a simple discipline: before granting any permission to an agent, ask what is the worst plausible outcome if the agent uses this permission in a way nobody anticipated, and size the boundary, and the human checkpoints around it, accordingly.

Readiness Signals

Questions worth answering before granting an agent real-world permissions.

Each of these can, and should, be tested and evidenced before deployment, not assumed and reviewed only after an incident.

Permission Boundaries
What is the agent authorised to do without a human check?

Defined narrowly, reviewed as scope changes, and sized against the worst plausible outcome of that authority being misused.

Auditability
Can you reconstruct what the agent did and why?

A tamper-resistant, system-level record of goals, steps and reasoning, independent of the agent being investigated.

Kill-Switches & Testing
Has the stop mechanism actually been tested?

A documented control that has never been exercised under realistic conditions is an untested assumption, not a working safeguard.

Where the honest answer to any of these is 'we assume so' rather than 'we have tested and evidenced this', that is the control to close before, not after, the agent is given real-world permissions.

04 · Action Logging and Auditability: Seeing What the Agent Did and Why
When something goes wrong, the organisation needs to reconstruct not just what the agent did, but the reasoning and inputs that led it to do so.

Auditability for agentic AI needs to go further than logging that an action occurred. It needs to capture the goal the agent was given, the sequence of intermediate steps and tool calls it took to get there, the information it used at each step, and, where possible, an explanation of why it chose that particular path over available alternatives.

This level of detail matters because agent failures are frequently not a single obviously wrong action, but a reasonable-looking sequence of individually defensible steps that combine to produce an outcome nobody intended. Without a full record of the sequence, it can be very difficult to understand where the reasoning actually went wrong, or to distinguish a genuine failure from a correct response to an unexpected input.

This record also needs to be tamper-resistant and independent of the agent itself. An agent should not be the sole source of the record of its own actions; logging needs to be captured at the system and infrastructure level, so that the evidence used to investigate an incident cannot be affected by the same system being investigated.

Practically, this requires investment in observability infrastructure purpose-built for agentic systems, capturing structured logs of goals, tool calls, intermediate outputs and final actions, retained for long enough and accessible enough to support both routine review and full incident investigation when required.

Organisations that treat basic action logging as sufficient are likely to find, at the point they most need it, during an incident, a regulatory enquiry, a customer complaint, that they can show what happened but not why, which is frequently the more important question.

05 · Kill-Switches and Escalation Paths
An agent operating faster than a human can review its individual actions still needs a reliable, tested way to be stopped the moment something looks wrong.

Because agents can execute multi-step sequences faster than a human reviewer can meaningfully track in real time, the ability to stop an agent quickly and completely, a genuine kill-switch, is a core safety control rather than an optional extra. This needs to work reliably under the conditions where it is actually needed: during an active, unfolding incident, not just in a calm test environment.

A credible kill-switch capability requires clarity on several things in advance: who has the authority to invoke it, how quickly it takes effect once invoked, what happens to actions already in progress when it is triggered, and how the agent's state and any partially completed tasks are handled and recorded afterwards.

Escalation paths matter just as much as the ability to stop an agent outright. Not every anomaly warrants a full halt; many warrant a pause and a human review of the specific action or decision in question. This requires agents to be designed with defined escalation triggers, situations, confidence thresholds or action types that automatically route to a human before proceeding, rather than only after the fact.

Both capabilities need to be tested, not just documented. An escalation path that has never actually been exercised, or a kill-switch that has never been triggered outside a controlled demonstration, is an untested assumption about how the system will behave under real pressure, precisely the moment it is most likely to matter.

Boards and risk committees should expect a direct, evidenced answer to a simple question for any agent with access to consequential systems: if this agent starts doing something wrong right now, how quickly can it be stopped, by whom, and how do we know that mechanism actually works.

06 · Testing Agents Under Adversarial Conditions Before Granting Real Permissions
An agent that performs well on expected inputs has been tested against the easy case. Real-world deployment routinely presents the hard one.

Standard testing for AI systems typically evaluates performance against expected, representative scenarios. For an agent that will be granted real permissions and real-world consequences, this is necessary but not sufficient. The more important question is how the agent behaves when it encounters ambiguous, contradictory, adversarial or simply unexpected inputs.

Adversarial testing for agents means deliberately probing for failure modes before deployment: prompts and scenarios designed to induce the agent to exceed its intended scope, combine permissions in unintended ways, misinterpret an ambiguous goal, or be manipulated by input from an external system or user into taking an action it should refuse.

This kind of testing needs to happen in an environment that resembles production closely enough to be meaningful, with realistic system access and realistic data, but without the actual ability to cause irreversible harm. Testing an agent purely in an idealised sandbox, disconnected from the messiness of real systems, risks validating behaviour that does not transfer to the conditions the agent will actually operate under.

It also needs to be an ongoing discipline rather than a one-off pre-deployment gate. As an agent's permissions, tools or the systems around it change, previously tested boundaries can shift in ways that reopen failure modes that earlier testing had specifically ruled out.

Granting an agent access to real systems and real consequences without this kind of testing is, in effect, choosing to discover its failure modes in production. Given that agent failures translate directly into real-world actions rather than reviewable content, that is a considerably higher-stakes way to find out where the boundaries actually are.

07 · Multi-Agent Systems and Compounding Risk
A single agent's failure modes are hard enough to anticipate. Multiple agents interacting with each other introduce failure modes that emerge only from the interaction itself.

As agentic AI matures, a growing number of deployments involve multiple agents working together, one agent delegating a sub-task to another, or several agents each responsible for part of a broader workflow. This introduces risk that does not exist, or is far smaller, in a single-agent deployment.

Individually well-behaved agents can produce problematic outcomes in combination: one agent's output becomes another's input, and small errors, ambiguities or edge-case misinterpretations can compound across the chain rather than being caught and corrected at any single point. The final outcome may not obviously trace back to any one agent behaving incorrectly in isolation.

Multi-agent systems also complicate accountability and auditability significantly. A full record of what happened has to capture the interaction between agents, not just each agent's individual actions, and identifying which agent's reasoning actually drove a problematic outcome can require reconstructing a genuinely complex interaction rather than reviewing a single decision.

Governance for multi-agent systems needs to account for this explicitly: permission boundaries and testing considered across the whole system of interacting agents, not agent by agent in isolation, and monitoring designed to catch emergent problems in the interaction itself, not only anomalies in any single agent's individual behaviour.

Organisations moving toward multi-agent deployments should treat this as a materially higher governance bar than single-agent deployment, not an incremental extension of it. The controls that were sufficient for one agent acting alone frequently are not sufficient once several agents are acting on each other's outputs.

08 · Board and Risk Committee Questions for Agentic AI
Boards do not need to understand the technical architecture of an agent to ask the right questions. They need a small number of direct, well-chosen ones.

Boards and risk committees overseeing organisations deploying agentic AI do not need deep technical expertise in how agents are built. They do need to ask a small number of specific, direct questions, and to expect specific, evidenced answers rather than general reassurance.

What is this agent actually authorised to do on its own, without a human confirming each step, and who decided that boundary. What is the worst plausible outcome if it uses that authority in a way nobody anticipated, and has that scenario actually been tested rather than assumed away.

How quickly can this agent be stopped if something goes wrong, who has the authority to stop it, and has that mechanism been tested under realistic conditions rather than only demonstrated in a controlled setting. What record exists of what the agent did and why, and is that record independent of the agent itself.

If this agent is one of several interacting agents, has the combination been tested, not just each agent individually, and is there a single accountable owner for the outcome of the overall workflow, not just for each component agent.

A board that can get clear, evidenced answers to these questions is in a fundamentally stronger position than one relying on a general assurance that appropriate controls are in place. Given how directly agent actions translate into real-world consequences, general assurance is no longer a sufficient basis for oversight.

Conclusion
AI that can only advise fails into a human checkpoint. AI that can act removes it. Governance has to be built for the second case, not the first.

Agentic AI represents a genuine step change in enterprise AI risk, not simply a more capable version of the generative AI most governance frameworks were built around. The defining difference is straightforward to state and easy to underestimate in practice: an agent's actions are consequences in themselves, not recommendations waiting for a human to act on them.

Traditional access control, built around predictable users with stable permissions, was not designed for a system that decides its own sequence of actions and can combine narrow, individually reasonable permissions into a capability nobody explicitly authorised.

Managing this risk requires purpose-built controls: permission boundaries scoped tightly and reviewed continuously, tamper-resistant action logging capable of reconstructing not just what happened but why, tested and reliable kill-switches and escalation paths, and adversarial testing that probes for failure before real permissions are granted, not after.

Where multiple agents interact, these controls need to extend to the system as a whole, since compounding failures can emerge from the interaction between agents that no single agent's individual behaviour would predict.

Boards and risk committees that ask direct, specific questions about authority, auditability and tested controls, and expect evidenced answers rather than general reassurance, are in a materially stronger position than those relying on confidence in the technology alone. As agentic AI capability continues to expand faster than most governance frameworks anticipated, that distinction is likely to matter more, not less.

Explore Related INTAI Services

Know What Your Agents Are Authorised to Do

INTAI helps organisations design permission boundaries, auditability, escalation paths and adversarial testing for agentic AI, before agents are granted access to real systems and real consequences.

Assess Agent Readiness Talk to Us
Scroll