Why Enhanced Investor Governance Can Curb AI Risk
Anthropic’s Dario Amodei
Over the last six months, AI companies have come under increased scrutiny. Just back in July, it was reported that research agents built by OpenAI escaped the isolated environment they were meant to stay inside during cybersecurity testing and breached the AI platform Hugging Face. According to MIT Technology Review's own count, more than a thousand agent instances were involved. Then, on 13 September, Anthropic chief executive Dario Amodei published an essay, 'We Must Pace the Frontier,' arguing that leading labs should deliberately slow their most capable models, the result of which saw OpenAI's Sam Altman, Elon Musk and Google DeepMind's Demis Hassabis backed him. That signal, from rivals of Amodei, got the markets to take note, with AI-linked and semiconductor stocks falling sharply the following Monday.
Coverage since has framed this as pace versus race, safety versus speed. Almost none of it has asked who actually holds the leverage to enforce a slowdown before the next funding round closes.
Eyes turned to governments around the world, some of which reacted with statements agreeing, while others saw the move as anti-competitive. But if governments are not reacting at the pace needed to manage what has been framed as a potential risk to humanity, who should be taking charge of internal governance at AI companies? Maybe the answer lies closer to home, with the investors who write the cheques, which raises a question most term sheets haven't caught up with yet: should AI investor governance start requiring the safety and oversight commitments that funding agreements currently leave out?
Why a public statement is not a governance mechanism
The fact is that when governments do not take charge or are too slow and bureaucratic to respond to issues, which can be a positive for companies who dislike regulation, then self-regulation becomes a way of safeguarding what has been presented as an existential threat. And the issue is that this is how AI is currently being framed, it has great potential, will transform our lives, but it can also kill us all. That is affecting how AI is perceived and the reputation of these companies and its founders and is worth not dismissing.
But a voluntary pact between four labs will not survive the pressure that created what is now seen as an arms-race-to-AGI. Whoever paces themselves loses ground to whoever does not, because the pressure to keep pace comes largely from the people funding the next round, from the markets stumping up the cash and the next round of investment, because AI, software and hardware, is cash-hungry.
In normal circumstances, Government regulation is the response most people reach for, and in this situation it does have a role to play. But as I said before, regulation and government intervention also moves at a trickle compared to the innovation pace of AI companies' newer models.
As an example, the EU AI Act became generally applicable on 2 August 2026, but its toughest provisions, including data governance requirements for high-risk systems, were pushed back further still, to December 2027 and August 2028, under a last-minute amendment the month before. In the United States, oversight remains largely voluntary. Waiting for legislation to catch up to a model release cycle measured in months is not a strategy, it is a hope.
A geopolitical scoreboard, not just a technology race
And if all this isn't already challenging enough, the investment and governance gap and the lack in appropriate regulation sits inside a wider one. AI has become a marker of national economic and strategic position, not only a growth sector, and the capital flows reflect it. Because, so far, we are talking about Western AI models and not China.
The US and Chinese figures diverge across trackers depending on whether they are measuring private investment, state-directed investment, or infrastructure capex. These are genuinely different things, and the difference matters more than the headline totals: the US model is overwhelmingly privately funded and lightly regulated, China's is state-directed and tightly controlled, and Europe is trying to buy its way into relevance with public money while building the most detailed rulebook of the three.
Is the West's own regulation actually the more protective one?
Here in the west, we are led and conditioned to believe, rightly or wrongly, that our own AI regulation is restrictive, and China's as permissive because it moves fast and enables scale. Yet, that doesn't really hold.
In December 2025, China's Cyberspace Administration released draft rules, the Provisional Measures on the Administration of Human-like Interactive AI Services, that goes further on user protection in AI companion products than anything currently in force in the West. Even Scott Galloway highlighted this in a recent Pivot episode.
The rules hold developers legally accountable for designs that encourage addiction or emotional manipulation, require repeated disclosure that users are interacting with a machine, and mandate break notifications during extended use. By contrast, explicit language addressing manipulation and dependency was removed from California's AI companion law during drafting.
And yes, this is a narrow and specific example, not a verdict on which regulatory system is better overall. China's broader AI framework remains built on state control of data and deployment, which cuts firmly the other way on other measures. But it's a useful corrective to the assumption that stricter equals Western and permissive equals Chinese. The real picture is more mixed than the shorthand allows, and mixed pictures are exactly where confident, evidence-led commentary earns more credibility than a settled narrative does.
That leaves the people who sit inside the incentive structure and can move at its speed: the investors who fund the next round, sit on the board, and set the terms.
Is investor governance keeping pace with the capital?
So, the scale of capital involved makes the question harder to avoid. Global venture capital hit a record $510 billion in the first half of 2026. OpenAI and Anthropic alone absorbed roughly 43% of it, about $217 billion between two companies, according to Crunchbase's H1 2026 data, AI's share of global VC dollars touched 80% in the first quarter of 2026 and settled around 70% in the second, up from roughly 50% a year earlier. Billion-dollar-plus AI funding rounds made up 86% of total AI investment in Q1 2026, against 55% in 2025 and 41% in 2024.
This isn't diversified exposure wearing a venture-fund label. S&P Global Market Intelligence reports that limited partners may be carrying 15 to 20% exposure to the same handful of AI companies across different fund managers, because general partners underwrite independently and do not manage an LP's aggregate exposure across the portfolio. Meanwhile, a persistent liquidity crunch, cumulative net cash flows to LPs have run roughly negative $200 billion since 2022, is pushing limited partners toward direct co-investment in AI names to chase returns without proportionally higher fees, according to PitchBook's Q2 2026 analysis. Median Series D+ AI pre-money valuations reached $4.7 billion in the first quarter of 2026, nearly four times non-AI peers. Capital is also consolidating: 91% of new LP commitments now flow to already-established VC firms, up from 74% a year earlier, while emerging manager fundraising has dropped to its lowest level since 2020.
None of this describes a market behaving irresponsibly. It describes a maturing asset class still working out its own governance norms while the underlying technology accelerates faster than most institutions are built to track.
The Center for Long-Term Cybersecurity at UC Berkeley has been running an active research project through 2026 specifically to build governance frameworks for investors and boards, on the premise that existing oversight and due-diligence processes were not designed for the risks frontier AI introduces. That project would not exist if the gap were not real and already visible to the people closest to it.
Looking at the processes that are already in place and what we see is that venture and private equity due diligence in 2026 has genuinely absorbed AI-specific compliance work, bias detection, data licensing, privacy law, and increasingly the EU AI Act's transparency requirements for high-risk systems. What it hasn't yet absorbed, in any consistent or structural way so far, is frontier safety exposure at the fund level, or a mechanism for a board seat to translate into a meaningful check on how fast a portfolio company is racing.
What the public actually thinks, and why the doom framing misreads it
Media coverage of this moment leans heavily on existential framing, AI as an unstoppable force racing toward catastrophe. The public data tells a more specific and more useful story on how these narratives are impacting how we the human public perceive AI.
Pew Research's February 2026 survey found that 52% of Americans are now more concerned than excited about AI, up from 37% in 2021, and about two-thirds believe the technology is moving too fast. Crucially, 60% of US adults said they are not confident that AI companies will develop and use these tools responsibly. A CNBC Generation Lab poll found a majority of respondents distrusted every single one of nine prominent AI industry leaders - Alex Karp, Peter Thiel, Dario Amodei, Sundar Pichai, Mark Zuckerberg, Sam Altman, Jensen Huang and Elon Musk tested - on whether they would act responsibly.
That is not fear of a hypothetical machine uprising. It is a specific, measurable judgement about the people and institutions building the technology, and their capacity for self-restraint. The doom narrative is easier to write and more dramatic to read, but it lets every actor with real leverage, investors included, off the hook. The question the public is actually asking, whether stated that way or not, is who has the power to slow this down and whether they are using it.
What investor governance could actually look like
None of this requires investors to become regulators, or to stop backing frontier AI. It requires the same discipline already applied to financial and legal risk to extend to safety governance, with terms that are enforceable rather than aspirational.
Three levers are already available and do not require new legislation to use:
Board-level safety reporting such as a standing item, not an annual disclosure, so a board seat carries real visibility into how a portfolio company is racing.
Staged funding is tied to independent audit access so capital continues to flow only where a company has agreed to external evaluation of its safety practices, not merely its financial performance.
Termination covenants written into funding agreements in advance, that give investors a genuine exit mechanism if a portfolio company's risk posture changes materially between rounds.
None of these are radical in any way, shape or form. Why? Because versions of them already exist in sectors where investors decided the downside of getting it wrong outweighed the friction of adding a clause. AI is arguably the sector where that trade-off is most obviously in the investors' own interest, reputational and financial risk in this space move together, not separately.
The fact is that if four rival chief executives are agreeing in public to pace themselves then there is a genuine issue and a signal that needs to be noted and accepted. The next meaningful signal will come from the people who write the cheques, and whether their term sheets start to look any different, which they should as investors are investing with private money they have and debt they are scaling up.
So the question now is, if governments who are historically slow in moving, then should it be up to investors and their term sheets to establish governance to protect not just their investment, but society as a whole? What do you think?