Governance as the differentiator
Governance maturity is an important differentiator between organizations that are seeing tangible benefits to AI adoption and those who have yet to realize these benefits. Good governance is not only about managing the vast landscape of risk around AI. More importantly, it creates trust, allows businesses to move confidently and decisively and speeds up adoption.
In practice, effective AI governance is challenging. AI platforms (including platforms to build AI agents) are increasingly democratized across an organization, resulting in an explosion in the volume of use cases. Each use case presents different risks and, therefore, there is no one-size-fits-all approach to AI governance.
The wider context, too, introduces complexity as the technology, geopolitics and regulation are constantly changing. So too are commercial drivers, with the combination of context-heavy agentic AI use cases and token-based pricing models making return on investment even harder to demonstrate.
And yet, one of the biggest risks to organizations is moving too slowly to maximize opportunities or fend off new AI-enabled competitors and disruption. Businesses should not let themselves be paralyzed by impractical, resource-heavy governance. Against this backdrop, the board’s role is crucial to calibrating governance such that it aligns with the organization’s AI strategy, risk appetite and resources.
The geopolitical, value chain and technological backdrop of AI sovereignty
AI is not a neutral technology. It is increasingly seen as a strategic asset that exists within a rapidly changing geopolitical, regulatory and market landscape. Jensen Huang, CEO of Nvidia, popularized the notion of “AI sovereignty” in late 2023, referring to a nation’s ability to produce AI using its own infrastructure and capabilities.
AI sovereignty, though, is a misnomer. No nation or business can purport to own its end-to-end AI capabilities. Dependencies are everywhere. Instead, AI sovereignty is about a nation’s or business’s ability to control the critical aspects of its AI capabilities against this complex, ever-changing backdrop.2
From a governance perspective, an organization’s understanding of its multiple roles in the AI value chain, including where it has control and where it has dependencies, is crucial to identifying, assessing and managing risk. AI sovereignty means having effective control over your AI deployment, remaining agile in response to changing business strategy and external shocks, and future-proofing investments.
Guided by its board, a business can consider AI sovereignty through three lenses.
Geopolitics and regulation
AI is subject to a complex and fragmented geopolitical and regulatory regime. Every nation is legislating differently to protect its economic and (because AI is inherently dual use) national security interests.
The two AI superpowers, the United States and China, are aiming for AI supremacy. Each is concerned with controlling access to its most powerful models, and export controls are evolving quickly and unpredictably, becoming a major focus for regulatory activity and tension.
The U.S. government has recently imposed ad hoc restrictions on certain advanced models and has announced a program for frontier labs to submit voluntarily to a new testing regime, details of which have not yet been published. This has been a shift from its previous non-interventionist approach and has caused businesses worldwide to reassess their exposure to regulatory change. In particular, AI cybersecurity is being viewed as a tool of state power, with access to key AI capabilities subject to state influence and control.
China is similarly considering export controls on its open-weight models, with growing interest globally into cheaper alternatives to U.S. frontier models. Domestically, China intends to boost adoption and economic growth through legislating aggressively and having sectoral regulators issuing guidance to businesses on AI development and deployment.
China’s recent AI transparency laws, as well as those passed in California, echo the transparency requirements set out in the EU AI Act. The EU, too, is concerned with sovereignty and has recently released a Tech Sovereignty Package, pushing businesses to “buy European.”
In the United Kingdom, the government has thus far maintained its light-touch regulatory regime, where sectoral regulators are expected to enforce their requirements in accordance with a set of high-level overarching principles.
The true AI regulatory landscape extends beyond AI-specific laws. Every body of technology-agnostic law is being challenged, reviewed and consulted upon to address AI-specific issues. Laws (and risks) relating to export controls, ESG, privacy, IP, antitrust, employment, consumer, as well as sector-specific requirements also apply.
In many jurisdictions, there are major policy initiatives around copyright, for instance, as economies seek to balance the interests of AI developers and creative industries. An enterprise governance model must manage risks across all of these inter-related areas. We discuss these further below.
The AI value chain
The AI value chain describes the interconnected layers of activity required to extract value from AI. Management teams and system owners should be aware of their business’s position in the value chain, as this will drive the legal risks, contractual considerations, regulatory requirements and the necessary governance steps.
Conceptually, most businesses will occupy the ‘squeezed middle’ between upstream vendors and downstream customers and end users. They will therefore need to consider how to manage upstream dependencies and maintain resilience to geopolitical, regulatory or commercial disruption.
To unlock value in proprietary data, robust data governance is required to ensure that privacy and wider data-related regulatory requirements are met, as well as contract protections against data leakage and infringement risks. Contracts are also a vehicle to extract value from data via (increasingly nuanced) data usage rights.
Almost every business will be developing and deploying AI applications, from fine-tuned open-weight models (at one end of the spectrum) to targeted retrieval-augmented generation-enhanced implementations or custom system prompts (at the other). Many of the risks and regulatory requirements bite here, and governance must be multi-faceted, spanning accountability issues, IP, cybersecurity and more. Customer-facing applications raise additional reputational risk.
The AI time horizon
AI technology is not static and therefore AI governance cannot be so. The technology is evolving rapidly, from AI models (which are becoming more capable at each new release) to agents, embedded AI, robotics, world models and possibly some notion of artificial general intelligence in the future.
Governance must support businesses’ changing AI strategy as it responds to these technological advancements. For example, many businesses are currently on the cusp of transforming their workflows using autonomous AI agents.
However, the use of agents raises specific risks. Hallucinations can cascade through systems, agents (being probabilistic machines) can break rules and guardrails, and agents can cause data loss and system failure even where no threat actor is involved.
The recent incidents involving AI agents escaping sandboxes and targeting third party infrastructure in pursuit of their objective illustrates that the risks of misalignment (long since espoused by AI safety researchers) are real and have potentially catastrophic effects. Boards must set the tone that significant investment is required in the governance frameworks to manage these novel risks.
Roles of the board and management
Against this complex backdrop, boards set the tone through oversight of their business’s AI strategy, risk appetite and allocation of key resources, while gaining an understanding of where and why AI is being used and focusing on commercially significant and higher-risk use cases.
In particular, boards should expect to be advised as to how controls are being implemented, and that governance is calibrated to match risk thresholds and the changing commercial and regulatory landscape. Reporting and metrics on critical use cases are useful. Ideally, boards should also treat AI as a recurring board agenda item. More broadly, management roles and responsibilities regarding oversight of AI may need to be reviewed, and this may require new escalation pathways or management committees.
A functional and effective relationship between the board and management teams is key. At the management level, AI strategy and risk appetite should be translated into practical limits, thresholds and controls. Governance should be sufficiently agile to meet the volume and democratization of AI and tiered to established risk thresholds.
In practice, tiering means that a low-risk use case, such as, for example, a new internal productivity tool for employees to use to summarize documents for their own internal use, may need no more than a referral to existing training and governance and inclusion in the AI inventory.
On the other hand, a customer-facing use case that incorporates agentic functionality would require, among other things, extensive pre-deployment testing, processes set up for ongoing monitoring and human review, and approval by a cross-functional committee before launch. The board’s interest is in confirming that the tiering criteria are coherent and that use cases are being classified appropriately.
It is helpful to maintain an inventory of material AI use cases with clear, responsible individuals for each use case and escalation paths to senior management and committees. Stakeholder teams involved in the design, operation and testing of use cases should be diverse in function and discipline. Regulatory compliance should be understood, managed and documented.
Boards should also start to develop fluency in AI, including how the different types of AI technology work, its benefits and limitations, and how to use it responsibly. Boards are increasingly exploring their own use of AI, for example, to assist with transcription and reviews and to support decision-making, and specific governance is required to ensure AI use does not compromise directors’ ability to fulfil their duties. Fluency is a continuous process and should keep pace with technological developments.
AI oversight is not a new category of duty for directors. It falls within directors’ existing obligations to exercise reasonable care and to ensure the business has information and reporting systems appropriate to its material risks. Where AI is commercially significant, the practical question for a board is whether it can demonstrate that it identified AI as a material risk, put a reporting system in place and responded to what that system told it.
Directors should also be alert to the three ways AI can create personal exposure: a failure of oversight where red flags were reported but not acted on, making statements about the business’s AI capabilities or controls that cannot be substantiated, and a failure to maintain the collective board literacy or independent assurance necessary to properly interrogate the company’s AI risks.
Board-level concerns across ai risks
Another key aspect of board-level AI fluency is understanding the legal risks that AI presents and the questions those risks raise for management. The categories below, including unpredictable black boxes, cyber, regulatory, third parties, and IP and data, are not discrete risks. They compound, particularly where agents are deployed at scale, which is why governance must be calibrated to risk thresholds rather than applied uniformly.
Unpredictable black boxes
AI models are probabilistic and, as a result, outputs can be inaccurate, inconsistent, or drift over time. As mentioned above, these risks are exacerbated when agents are used. Boards should be aware that new governance and risk models are needed to manage the proliferation of use cases and agents across a business.
Errors can lead to significant financial and reputational impact, especially when AI is deployed at scale. If consumers are directly affected, regulators may take action and, for some use cases (e.g. involving large employee or customer bodies), there is a risk of class actions in certain jurisdictions.
For a board, a key concern will be ensuring that management teams have put in place governance to assess the risk profile of use cases, that there is adequate oversight of the highest-risk use cases, and that safeguards and controls are in place to manage, detect and respond to errors and undesirable agent behaviors.
Cyber
Cyber is one of the most significant AI risks. AI is being used, offensively and defensively, to accelerate the identification and exploitation of cyber vulnerabilities, and to conduct all stages of an attack. AI models can be targeted maliciously, such as through prompt injection attacks (i.e., malicious input used to manipulate a model), and AI-driven attacks can outpace existing detection, patching and response capabilities.
The proliferation of AI tools, integrations and agents increases the possible points for attack.
Most concerningly for boards, the use of autonomous agents increases risk as agents may take high-impact actions (e.g. accessing, disclosing or deleting data beyond their intended scope) without an external attacker being involved.
Cyber incidents can cause significant data loss, service disruption, operational downtime, regulatory intervention, and litigation. Information security and AI safety researchers have long maintained that AI agents are mathematically impossible to secure, so the governance steps are about risk management rather than elimination.
Boards should make AI-driven cyber risk a priority and challenge management teams as to whether the business is modernizing its cyber capabilities to ensure speed of vulnerability management, detection and incident response, and there is adequate governance around AI agent deployments.
Regulatory
As mentioned above, there is growing fragmentation of AI regulation globally, with every nation legislating differently to protect interests and governments acting unpredictably. As a national security asset, AI is subject to increasing government intervention. AI superpowers, the United States and China, are competing to develop models and curtail each other’s access to these models. M&A activity has also been unwound or blocked under foreign investment regimes globally.
Businesses are subject to a high compliance burden across AI laws, export controls, privacy, IP, cyber, operational resilience, antitrust, employment, financial regulation, and consumer laws. Some of these (for example, the EU AI Act and GDPR) give regulators powers to impose substantial fines for AI breaches that affect the bottom line. However, many of these issues have not yet been tested.
There is an early but mounting wave of enforcement actions, investigations and disputes related to antitrust, copyright, personal injury, product safety and discrimination. Regulators globally are prioritizing AI for enforcement, including around price collusion, market concentration, data protection, human rights and safety. We are likely to see a torrent of commercial disputes around AI in the near to medium term.
Boards will be keen to ensure that management teams have a strategy to cope with external shocks and divergent regulatory regimes. The business’s risk management framework must be capable of withstanding regulatory scrutiny and any external messaging should align with compliance realities. Some businesses may choose to engage early with key regulators prioritizing AI and identify and manage exposure to regulatory action.
Third parties
Most businesses will depend on third parties for critical AI capabilities (models, compute, applications and data). As mentioned above, this can result in concentration risk. Businesses may become locked into certain vendors or certain standards (e.g. for agent-to-agent communication), making it costly and complex to switch providers at a time when flexibility is crucial.
Given the commercial landscape, negotiations with AI vendors are complex. Conflicts of interest are rife in the AI value chain and AI vendors may optimize for their own commercial interests rather than their customers.
In addition, allocating liability across the AI value chain is increasingly challenging, particularly as the potential losses increase with deeply embedded agentic AI systems.
The question of accountability for AI harms has two dimensions: the position at law and the position under contract. For many use cases, current laws are not fit for purpose, and most businesses will need to rely on their contracts to allocate risk. However, contractual risk allocation is far from a panacea. As businesses scale adoption, they will likely become exposed to significant liability for adverse outcomes despite having limited control over the upstream models on which they depend.
Boards will want to confirm that management teams understand the business’s third-party dependencies and the impact of disruption, and have taken steps to ensure resilience. Businesses should ensure diverse partnerships across AI companies, model types and geographies, and that terms with vendors protect assets and allocate risk fairly. Boards should ask whether the business is carrying risks under customer agreements that are not adequately mitigated under its upstream vendor agreements (or vice versa).
IP and data
There are multiple IP risks around building and using AI. In relation to IP ownership, copyright (which is jurisdiction-specific) is unlikely to subsist in AI-generated works (albeit this is a very nuanced and technical question, and the subject of a lot of policy debate globally).
In principle, the upshot is that where AI has been used to generate commercially significant assets that would traditionally have been protected by copyright (e.g. code), there may be significant degradation in IP protection.
In relation to IP infringement, third-party IP may be infringed both in development and deployment. Injunctions and damages claims for IP infringement can halt revenue and significantly impact the bottom line. Boards will want to ensure that management teams have a strategy to preserve the value of proprietary AI innovations and manage IP infringement claims across relevant jurisdictions.
From a privacy perspective, key principles (e.g. accuracy, transparency, purpose limitation and data minimization) are inherently in tension with how AI systems work. AI risks are a key focus for data regulators globally, particularly in respect of AI agents.
For a business, privacy concerns can impact customer trust and employee morale and raise the prospect of regulatory intervention. In addition, biased data or design can produce systematically discriminatory outcomes and can risk reputational damage and class actions.
Boards should consider whether management teams have put in place controls to prevent privacy and bias concerns escalating in customer-facing deployments.
Our take
Ultimately, a successful AI adoption strategy must be underpinned by good governance that works with (and not against) the business and the evolving technology. However, a coherent, agile and effective governance framework requires investment. It is not sufficient to rely on a patchwork of existing policies and processes, which can expose the business and will likely prove to be a false economy. The investment must be made in people who are a business’s first and last line of defense, and therefore, continuing education and training should be a priority.
Footnotes
1. KPMG’s Global AI Pulse Survey, Q1 2026.
2. See AI as a strategic asset: what boards need to know about AI sovereignty to stay ahead.