Three-quarters of company executives surveyed as part of a new IBM report expect AI agents to have some level of autonomy in making decisions within their organisation by 2027. But only 37% say their current governance is designed to scale in line with the growth of agentic AI.

As AI agents become more prevalent in enterprises, taking on more tasks and making more decisions autonomously, governance and accountability become even more important. What is the state of play of AI governance today and what are the pressing issues and gaps in this area as agentic AI begins to be more widely adopted?

These are some of the issues raised in the report, The New Realities of AI Governance: What the Rise of AI Agents Means for Governance and Accountability in EMEA, which is based on a survey of 3,221 senior executives at organisations across eight countries in Europe and the Middle East.

Here are five important questions explored by the report:

1. Will AI governance keep pace with agentic AI?

80% of company executives said that their board or C-suite had been made aware of an AI related incident affecting the organisation in the last 12 months. This includes incidents such as agent failure, data exposure, AI-driven decision error, or AI security breaches. Despite board level decision makers now being more aware of AI problems like this, only 40% of those organisations reported a change in AI governance policies or frameworks as a result of an incident.

While acknowledging that increasing senior level awareness of AI issues is a positive, the report suggests that greater attention by executives needs to produce consistent policies, processes and follow-ups when AI causes problems. This becomes even more important as agentic AI enters the mainstream.

Already, over a third of organisations surveyed say they are actively using agentic AI in some of their live processes. By 2027, 75% expect agents to make decisions and act without human approval of every action.

The freedom with which agents are allowed to act should develop at a pace that enables the governance of those agents to keep up.

You obviously can't require every agent action to be approved by a human, otherwise you would quickly wipe out the efficiencies of AI. Enterprises need to set clear boundaries and rules dictating agent activity, monitor outcomes and manage exceptions where agents step over the boundaries.

Start by introducing agents in well-defined, lower-risk processes, the report suggests, and be clear about what each agent can do, and when it must involve a person. Runtime controls should be set up in line with the risk of each process, so that actions can be monitored to ensure policies are being followed or stopped in real-time if needed.

2. Who is accountable when an agent does something wrong?

While nearly all organisations in the survey have assigned responsibility for AI risk at board level, there is a lack of agreement on who is accountable if an AI agent does something wrong.

For example, 17% say accountability should rest with the CIO or CTO, while 14.5% say it should be shared across the C-suite. Others say it should be down to the deployment team (11%), the AI vendor (10%) or the line-of-business leader who sponsored the AI system (10%). 9% say the CEO should be held accountable.

When it comes to who is legally liable, views are similarly divided. One in five (20%) say the liability should be shared between the enterprise that deploys the agent and the vendor. Others say the issue is too complex for a general answer (19%) or that the necessary regulatory framework does not yet adequately exist (19%).

Lack of clarity on who is accountable or liable can mean the enterprise is slower to respond when AI makes a mistake or it can cause internal conflict. Ultimately, uncertainty in these areas can slow AI adoption.

While opinions differ on who should be held accountable or liable, executives point to several capabilities that companies must have in place as agent autonomy increases.

33% think there should be formal vendor-liability frameworks to document the extent to which vendors can be held accountable for AI products and services. 32% agree there should be defined thresholds for when an action or decision by an AI agent should be escalated to a human; 30% expect real-time monitoring and audit trails for AI agent activities and 29% expect companies to have the ability to pause agent activity across the enterprise.

3. How far should governance and controls extend to third parties/partners?

AI governance does not stop at the AI systems deployed within an organisation's boundaries. What about governing the risks of AI used by its vendors, partners and other third parties it collaborates with in the wider supply chain?

More than half of executives (54%) in the survey admit that there are gaps or limitations in how their governance covers AI used by external businesses they work with.

For example, AI risks can be introduced not only by the behaviour of individual AI agents inside the enterprise, but also from their interactions and data exchanged with agents used by business partners and other third parties.

Currently, contractual provisions (43%) and audit rights (39%) are two of the main ways companies use to control external AI like this. 21% use continuous monitoring of third-party AI while 13% insist on independent assurance or certification of any AI systems used by third parties.

While contracts and audits can form part of the solution, the report's authors suggest they may need to be underpinned by ongoing visibility into third party AI agent behaviour and runtime controls that can stop agents acting if they step out of line.

4. Should governance costs be included when measuring the ROI of AI?

Having adequate governance processes and systems for AI is an essential requirement for enterprises that want to deploy AI responsibly. So, governance should be included in the way the ROI for any AI deployment is measured. But not all enterprises include it.

The report suggests that most (87%) organisations now have a formal framework for measuring AI's ROI, but only 40% actually apply it across the enterprise. And within organisations with an AI ROI framework, nearly one in three (32%) don't include governance costs in their measurement.

To accurately measure the ROI of AI, businesses need to understand both the benefits they can expect from AI and the costs and resources required to manage its risks. So, governance costs are a legitimate consideration that needs to be factored into the ROI calculation for any AI system.

5. What are the governance-related barriers to AI success?

According to the research, executives are divided on the most significant governance-related barriers to achieving a return on investment in AI over the next 18 months.

The most often cited barriers were the use of governance frameworks that were designed for an earlier era of AI (16%) and data-sovereignty constraints (16%) (i.e. regulatory restrictions on where a company's data can be stored, processed and accessed).

Regulatory uncertainty came a close third (15%) unsurprisingly given the complex nature of AI and the difficulties that regulatory bodies face in trying to keep up with the pace at which AI is evolving.

14% mentioned inadequate liability frameworks (if its unclear who is liable when AI makes mistakes, companies may stick to less risky and potentially lower value AI use cases) and 13% pointed to the difficulties that enterprises have in measuring and attributing AI's return on investment (if you can't measure returns, it's difficult to make accurate investment decisions).

As agentic AI becomes more widespread, the enterprise's ability to deploy and scale AI agents effectively across the organisation will become an increasingly important driver of business success. Just as important, however, will be the ability to monitor, understand and govern agent activity.