Reference entry

Agentic AI risk

Agentic AI risk is the possibility of harm or loss arising when an AI system can take actions towards a goal with limited step-by-step human direction. The risk grows when a system can use tools, change data, communicate with other systems or delegate work to other agents.

An agent may plan a sequence of steps, call software tools, retrieve information or act on a user’s behalf. This can make a task more efficient, but it also expands the system’s authority. A mistaken interpretation that would be harmless in a text answer can have greater consequences if the agent can send a message, alter a record, make a purchase or launch another process.

Delegation changes the risk boundary

Agent-to-agent delegation can make it harder to see which system made a decision and what instructions shaped it. Each hand-off may change the context, permissions or level of checking. An organisation should map the chain of agents and tools, constrain each actor’s access to what it needs, and keep records that make actions attributable. It should also test how the chain behaves when an agent receives bad data, an ambiguous goal or hostile instructions.

When several agents interact, their combined behaviour may not be predictable from testing each one in isolation. Agents can repeat actions, reinforce errors, compete over shared resources or pass along flawed assumptions. This multi-agent emergence is a system-level concern: monitoring only one model or one user interface may miss activity taking place across the connected workflow.

Oversight must work at system level

Human oversight is meaningful only if a person can understand when intervention is needed and can act before harm becomes difficult to reverse. Review points should reflect the impact and reversibility of an action. Low-impact steps may be automated within clear limits; consequential or external actions may need confirmation, dual approval or a restricted operating mode.

A kill switch should be designed and tested as part of the system, not left as an informal promise. Teams need to know what the switch stops, whether it revokes tool permissions and queued tasks, and how to restore service safely. A named owner should be able to pause the agent, while incident responders can preserve records and investigate what happened.

Before deployment, organisations can start with narrow permissions, sandbox testing and explicit action limits. During use, they should monitor not only output quality but also tool calls, delegation patterns, changes in task scope and attempts to exceed authority. Risk review should repeat when the model, tools, permissions or operating environment changes.

Go deeper: governing the agentic enterprise