Anthropic's newest AI tools are making customers decidedly nervous

Key topics:Mythos AI is powerful and risky; limited release, tight controlAnthropic balances safety image with IPO push and global outreachMore autonomous AI tools raise opacity and accountability concerns.By Parmy Olson.Catching a glimpse of Dario Amodei these days is like finding a rare butterfly. The chief executive officer of Anthropic PBC was scheduled to meet with 50 fellow bosses from some of Europe’s largest companies in Oxfordshire last week, an exclusive convocation by his firm at a Jacobean-style mansion that included former British Prime Minister Rishi Sunak, now an Anthropic adviser. Amodei had been scheduled to travel to another part of Europe afterwards, but his trip was cut short to just one or two days in the UK. Why is he so hard to pin down? Because of Mythos, the artificial-intelligence model he teased last month as too dangerous to launch. Anthropic staff have access to the technology and say it is more human than its forerunners. But it could also be weaponized for cyber attacks, so the company has let a few dozen organizations test and preview it. The model’s power and potential will help Amodei’s efforts to repair relations with the Trump administration, and he’s eager too for Mythos to finally reach members of the public. That requires a juggling act that, for now, Amodei is managing to pull off. .Read more:.Why the US's current IPO mania could signal top of stock market.While his company races to get Mythos market ready (and itself ready for a stock-market listing), it’s also publicly positioning itself as a responsible societal actor. It is meeting with global institutions to help them “absorb the shock” of its technology, according to one Anthropic insider. That takes in closed-door briefings with the Financial Stability Board in Switzerland to discuss potential Mythos cyber risks, and joining the pope this week as he announced the Catholic church’s position on AI, including warning against its use in warfare.That Anthropic was sending a co-founder, Christopher Olah, to join Pope Leo while simultaneously suing the Pentagon over how its technology can be used in war is a remarkable signal of its ability to be two things at once.Anthropic has always been a walking contradiction. It brands itself as the safest of AI companies while also racing to build super-intelligent machines whose impact on the world is fraught and untested. Its restructuring as a public-benefit corporation has given it the virtuous glow of a firm with principles, even as it prepares to cash in on the AI boom with an initial public offering that could be risky for retail investors. It has given chatbot Claude a moral constitution to guide behavior, but also endowed it with unnerving warmth and sycophancy that can encourage dependence from human users. There was more incongruity at the company’s confab for developers in London last week, when hundreds of engineers from industries such as finance and energy as well as startups crowded into a warehouse-style venue to get the latest on its popular programming tool, Claude Code. Several workshops showcased its growing autonomy, including the ability for tens or hundreds of AI agents to continuously run tasks for hours on end, unprompted. .Read more:.Microsoft could be Sam Altman's secret IPO weapon, but OpenAI boss ignores it: Parmy Olson.One developer from a large Wall Street bank told me that made him uneasy. The more independence that’s given to AI tools, he reasoned, the harder it will be to follow a chain of accountability if something goes wrong. He wanted to see more levers to control the technology. When I put those concerns to Cat Wu, head of product for Claude Code, she said the system was “incredibly secure” and there was “more of a problem with us explaining how we build security into the product.” The issue, in other words, was one of communication, not necessarily of controls. Wu said Anthropic’s models were aimed at becoming more independent and proactive over time. “Mythos is a model that exhibits this,” she added. So while workers in regulated, systemically important industries such as finance might like more human control, Anthropic is pushing its technology toward needing less overall. Although many developers at the conference said they liked coding with AI, some were unsettled by how removed they were becoming from the process and how the tool seemed increasingly like a black box.One pointed out that a year ago, Claude Code would automatically show a stream of text detailing the way it was processing each step of a task. Now that is hidden away by default and has to be sought out, and that more opaque output has drawn complaints from other developers. The coders at Anthropic’s conference didn’t appear to care, but this is more evidence of humans taking their hands off the controls. They had come to trust Claude enough to not check its method of working things out..Read more:.Revolut leads the pack: Fintech IPO showdown with Klarna and Monzo - Paul J. Davies.Anthropic’s public work in courting institutions like the FSB and the Vatican, as well as advertising for staff to help it partner with development banks and UN agencies, will probably become ever more necessary as its tools are imbued with greater autonomy. The more they advance, the more the world is being asked to trust the company. As Amodei races against his competitors to get Mythos safely over the line, that juggling act could get harder to maintain..© 2026 Bloomberg L.P..Sign up for your early morning brew of the BizNews Insider to keep you up to speed with the content that matters. The newsletter will land in your inbox every morning on weekdays. Register here.Support South Africa's bastion of independent journalism, offering balanced insights on investments, business, and the political economy, by joining BizNews Premium. Register here.If you prefer WhatsApp for updates, sign up to the BizNews channel here.

Anthropic's newest AI tools are making customers decidedly nervous

Related Stories

The Economist: Deep mutual distrust will scupper US-China co-operation on AI

Martin Wolf: With AI, "I told you so" will be too late

FT: AI automation is starting to carve up the gig economy

OpenAI 'solves' a Millennium Problem — and 24 of world’s best Maths brains cry foul