When AI’s builders sound the alarm

Betania Allo examines why frontier AI companies are warning of the risks of greater capability, and what the shifting economics of AI mean for security, sovereignty and critical infrastructure in the Middle East.

Something unusual is happening inside the frontier AI industry. The companies that have spent years racing to make artificial intelligence more capable are beginning to talk about slowing down. The timing matters for a region rapidly integrating AI into cyber defence, critical infrastructure and national security environments, where greater capability can bring new dependencies into systems designed to remain resilient under attack.

On 12 September, Anthropic CEO Dario Amodei called for the industry to moderate the pace of capability development. Sam Altman backed the idea and confirmed that OpenAI would not proceed with an IPO in 2026, describing the current moment as unsuitable given mounting safety concerns. The announcements followed warnings from former employees, increasingly powerful cyber capabilities and documented cases of AI systems acting beyond intended constraints.

The politics of the warning

Taken literally, this looks like a safety story. But it is also becoming a story about the political economy of artificial intelligence.

Daniel Innerarity offers a useful way into the problem in A Critical Theory of Artificial Intelligence. Reflecting on the 2023 proposal for a six-month moratorium on advanced AI development, he questioned both its practicality and the authority of the people making the most dramatic predictions. Expertise in building AI systems, he argued, does not automatically confer expertise over the future of society. Apocalyptic scenarios can enlarge perceptions of technological power while directing attention away from the institutions, incentives and political choices surrounding it.

When the risks become real

Three years later, the warnings deserve more weight because some of the risks have become considerably less hypothetical. Anthropic’s latest threat intelligence reporting documents the use of AI in cyber operations, fraud, weapons-related research and other forms of abuse. OpenAI and other frontier laboratories have encountered models capable of discovering vulnerabilities and operating with increasing autonomy. Security professionals have, therefore, little reason for complacency.

Follow the money

Yet the timing also deserves scrutiny. Frontier AI is extraordinarily capital intensive. OpenAI has contemplated around $600 billion in compute expenditure through 2030, while Sam Altman has discussed infrastructure ambitions measured in trillions of dollars. These companies have been able to sustain extraordinary investment because private capital has been willing to finance growth on the assumption that tomorrow’s AI economy will justify today’s spending.

That assumption is now facing pressure from two directions. The first is the prospect of public markets. An IPO does more than provide another pool of capital. It changes the institutional environment around a company. Financial performance becomes visible in far greater detail. Investors scrutinise margins, infrastructure commitments, customer concentration, revenue quality and the path to sustainable profitability. The spectacular valuations attached to frontier AI eventually have to confront ordinary questions about cash.

Anthropic is moving toward a potential Nasdaq listing. OpenAI, however, has postponed its own offering rather than abandoning the possibility altogether. Once companies of this scale enter public markets, some of the opacity that private financing permitted becomes much harder to maintain.

The price of intelligence

The second pressure may ultimately be more disruptive: the price of intelligence itself is falling. Chinese developers including DeepSeek, Alibaba’s Qwen, Z.ai and Xiaomi are releasing highly capable models at radically lower prices, often with open weights that allow organisations to deploy and adapt them independently. DeepSeek currently prices some inference at well below one dollar per million tokens. Alibaba offers Qwen models at similarly aggressive rates, while Xiaomi has released its MiMo models under a permissive MIT license and has distributed enormous volumes of free tokens to developers.

This changes the competitive equation. Western frontier labs cannot simply charge whatever their immense development costs might appear to require. If an enterprise can obtain sufficient performance from Qwen, DeepSeek, GLM or MiMo at a fraction of the price, proprietary providers have to compress their own prices or demonstrate that the performance difference produces measurable economic value.

And OpenAI is already responding. Its CFO recently said the company cut the price of its Luna model by 80 percent, triggering a tenfold increase in usage, while arguing that OpenAI can now compete economically even against some deployments of Chinese open models.

This is why saying that consumers are simply “underpaying for tokens” would be misleading. Tokens have no fixed intrinsic value. Their market price is being discovered in real time, while the marginal cost of inference falls and aggregate spending on compute continues to rise. Frontier companies face a peculiar economic problem: intelligence is becoming cheaper to consume at the same moment that remaining at the frontier requires staggering amounts of capital.

Economics become governance

That is the tension that matters for governance. A company facing enormous infrastructure commitments, aggressive low cost competitors and future shareholder scrutiny does not make safety decisions in an economic vacuum. Those incentives travel downstream. When the same models are embedded into security operations or critical infrastructure, decisions about release, access and autonomy can become operational dependencies for institutions that had no role in setting them.

The existential warnings we see can be sincere. No doubt that their political consequences remain significant. When the companies developing the most powerful systems tell society that those systems may become catastrophically dangerous, they simultaneously establish themselves as privileged interpreters of that danger. They build the technology, measure its capabilities, define internal risk thresholds and propose many of the mechanisms through which governments should supervise them.

Innerarity’s concern becomes especially useful here. Technical expertise can quietly acquire political authority. Public shareholders will ask questions that safety frameworks do not answer. Open-weight competitors will constrain prices. Customers will compare outcomes rather than model prestige. Governments will increasingly treat compute, models and data centres as strategic infrastructure.

A different model for the Gulf

The Gulf may provide an early test of a different economic model. Saudi Arabia and the UAE are building sovereign AI ecosystems at the same moment that the economics of frontier models are being challenged elsewhere.

HUMAIN is particularly interesting because it does not depend on a single model winning the global intelligence race. Backed by Saudi Arabia’s Public Investment Fund, it is being built across the entire AI stack, from data centres and cloud infrastructure to models and enterprise applications. If inference continues to become cheaper, that structure may become an advantage. The value can migrate away from the token itself and towards the infrastructure, orchestration, proprietary data and applications surrounding it.

The region is also becoming a model producer. Saudi Arabia has ALLaM, while the UAE has developed Falcon and Jais. Falcon is particularly relevant to the economics of the current debate because TII has pursued efficient and openly available models capable of competing with much larger systems. In other words, the pressure on the American frontier-lab model is not coming only from China.

This creates an unusual strategic position for the Gulf. It can buy American frontier capability, deploy increasingly competitive Asian and open models, develop Arabic-first intelligence and own much of the infrastructure on which all of them run. Sovereignty therefore starts to mean more than hosting data locally. It becomes the ability to change models without surrendering the surrounding stack.

Sovereignty is a security issue

For the Middle East, this makes sovereign AI a security issue as much as an economic one. Governments are becoming investors, infrastructure owners, regulators and major users of AI while deploying it across increasingly sensitive environments. Owning more of the stack can reduce external dependencies, but it also concentrates responsibility for securing the infrastructure and governing what models are allowed to do within it. Model risk cannot be separated from the architecture through which that model is deployed.

The industry may indeed need to slow down. But what happens when economics eventually force it to balance itself? For years, private capital allowed frontier AI companies to pursue capabilities ahead of proven profitability. Open-source competition is narrowing that freedom from one direction. And capital-market discipline may narrow it from another.

The darker the predictions become, the less governance can depend on the assurances of those building the systems. If frontier labs believe the risks are as serious as they say, authority over deployment and intervention has to extend beyond the companies themselves. There is where the doom narrative becomes a governance issue.

Visited 21 times, 1 visit(s) today