The Economist: Should AI labs unquestioningly obey the Pentagon’s orders?

Key topics:Pentagon threatens Anthropic over AI battlefield useAI firm resists mass surveillance, autonomous weaponsRivals like xAI and Google move deeper into defence AI.Sign up for your early morning brew of the BizNews Insider to keep you up to speed with the content that matters. The newsletter will land in your inbox every morning on weekdays. Register here.Support South Africa's bastion of independent journalism, offering balanced insights on investments, business, and the political economy, by joining BizNews Premium. Register here.If you prefer WhatsApp for updates, sign up to the BizNews channel here..From The Economist, published under licence. The original article can be found on www.economist.com© 2025 The Economist Newspaper Limited. All rights reserved..The Economist.Pete Hegseth, America’s secretary of war, is taking a my-way-or-the-highway approach to the use of artificial intelligence on the battlefield. On February 24th he gave an ultimatum to Anthropic, maker of the Claude family of models: if it did not agree to terms set by the Pentagon on usage of its AI for defence purposes, it would face severe penalties. It is not the first time the Trump administration has publicly picked fights with companies that fail to follow its orders. In this case, though, Anthropic has leverage.The showdown took place during a meeting at the Pentagon between Mr Hegseth and Dario Amodei, Anthropic’s boss, whose credo is “Responsible AI”. Mr Amodei was summoned to the Department of War (DOW) because Anthropic is in a unique position. Among AI labs, it was the first to do classified work for the Pentagon, via a partnership with Palantir, a data firm, and Amazon Web Services, a cloud provider. But it also has clear red lines when it comes to the use of its models for national security. In negotiations with the DOW, it has insisted that Claude be used neither for mass domestic surveillance nor for building autonomous weapons..Read more:.The Economist: AI threat to Google/Meta is one thing; is digital advertising really recession-proof?.The restrictions have put it at loggerheads with Mr Hegseth, who has stipulated that firms providing the Pentagon with AI models must give it carte blanche to do with them what it likes when used for lawful military actions. In the past week, the DOW put its entire relationship with Anthropic under review, according to a spokesman. At the latest meeting with Mr Amodei, Mr Hegseth dialled up the rhetoric, vowing to terminate Anthropic’s contract by February 27th if the AI lab did not agree to the Pentagon’s terms, according to sources familiar with the discussions. A senior Pentagon official said that if Anthropic did not “get on board” with the DOW, the latter would invoke the Defence Production Act (DPA), a law that gives the president authority to oblige companies to do national-security work, as well as labelling Anthropic a supply-chain risk. (Anthropic understood this to be an either/or threat.)Anthropic’s main contract with the DOW is worth no more than $200m, a trifling sum for a firm that generated an annualised $14bn of revenue in February. But it cannot take the standoff lightly. Stripping Claude out of the Pentagon’s supply chain would have a big impact, given the large number of companies that do defence work. It is a punishment usually meted out to companies linked to hostile powers. The DPA has been invoked in recent emergencies such as the covid-19 pandemic. It is rarely brandished in such an adversarial way.That the Pentagon is threatening these additional measures against Anthropic, however, indicates that the administration faces a quandary. The DPA threat suggests that it is reluctant to rip Claude out of defence work. According to former defence officials with ties to Silicon Valley, this is because Anthropic is one of the best of only a few AI model-makers, which may make it indispensable to war-fighters.Will the standoff create an opening for rivals with fewer qualms? OpenAI, maker of ChatGPT, has been slower to seize the opportunity to work with the DOW. Its models are used by Microsoft, with which it was once joined at the hip, for highly classified defence work, but OpenAI is not a party to the contract. Some contestants in a competition to build voice-activated drone-swarming technology for the Pentagon are using OpenAI’s models, but again its involvement is indirect. Its only formal contracts with the DOW are for unclassified work, and the use of its models for national-security purposes is considered on a case-by-case basis.Fears of militarising AI run deep at Anthropic and OpenAI. At least until recently, both had safeguards against using AI to make weapons (the DOW has demanded that these be scrapped). The pair are also alert to the risk of losing their brainy AI researchers, many of whom come from abroad and may not share the Trump administration’s ideology.By contrast, Elon Musk, who previously warned against “killer robots”, appears to have shed his compunctions. SpaceX, his rocket and satellite company, and xAI, the model-maker with which it is merging, are reportedly competing together in the Pentagon contest to make drone-swarming technology. Grok, xAI’s model, is “on board” with being used in classified settings, the Pentagon official said.Google, another leading AI developer, is also taking on contracts for classified and unclassified work with the Pentagon, having scrapped restrictions on the use of AI for defence purposes in 2024. That is a striking reversal for the tech giant. It was forced in 2018 to relinquish a Pentagon contract called Project Maven, which used machine learning to analyse footage from drones, after an internal revolt.The Project Maven saga carried lessons both for Silicon Valley and the Pentagon that are worth remembering, say former defence officials. For tech firms, it may be unrealistic to think that they can control how their technologies are used on the battlefield. They can urge caution, but it is constitutional oversight of the armed forces that ultimately determines how wars are fought.For the DOW, however, demanding unfettered access to technologies with the potential for extreme lethality requires building a bedrock of trust. That can be eroded if these technologies are used for actions of dubious legality. The former defence officials say controversial decisions such as strikes against civilian drug-smuggling boats in the Caribbean raise concerns about how autonomous weapons systems could be misused in the future.Since the Project Maven days, the mood in Silicon Valley has become more pro-Pentagon. Many defence-tech firms have welcomed Mr Hegseth’s efforts to “accelerate like hell” and enlist newcomers to create military tools such as drone swarms and AI agents. But if he destroys this nascent trust with heavy-handedness, he may jeopardise his access to more than just Claude.

The Economist: Should AI labs unquestioningly obey the Pentagon’s orders?

Key topics:

From The Economist, published under licence. The original article can be found on www.economist.com

© 2025 The Economist Newspaper Limited. All rights reserved.

The Economist

Related Stories