AI & SocietySeptember 25, 2026

The Anatomy of a Hype

Sahra-Josephine Hjorth lying among yellow buttercups in a purple dress, her bodice opened to reveal a glass heart with clockwork gears, a red poppy and a house of cards

How AI extinction became a pre-IPO story

If Anthropic told prospective investors that its lead was narrowing, that customers were routing work to cheaper models and that prices were falling, it would have an IPO problem.

If it told them that the technology had become so powerful it might kill us all, it would have an AI safety problem.

Only one of those problems supports a $2 trillion valuation.

On September 8, Anthropic researcher Jacob Coxon resigned with an accusation aimed directly at the companies building frontier AI. After three years doing pretraining research at OpenAI and Anthropic, he wrote that neither was acting responsibly and that both were racing toward self-improving superintelligence and "gambling with our lives." His posts reached tens of millions of people within a day. Anthropic did not dispute him. One of its own alignment leads agreed with him in public.

Every anatomy lesson is a performance

On 31 January 1632, a man named Aris Kindt was hanged in Amsterdam for robbery, and his body was given to the Guild of Surgeons for its annual public anatomy lesson. The guild permitted only one public dissection a year. The praelector, that year Nicolaes Tulp, cut and lectured while students and members of the public paid to watch. Rembrandt painted the occasion, and the surgeons gathered around Tulp contributed to the fee to be included in the picture.

The painting is faithful to almost everything except the procedure. A real public dissection began with the abdomen, because the bowels were the first part of the body to decay, and only then moved to the head and limbs. Rembrandt shows Tulp holding up the muscles of the forearm, the most elegant thing in the body to display. And no one in the painting looks directly at the corpse. They look at Tulp, past him and out of the frame.

What happened in September followed the same script. A real body was opened in public by the only people permitted to cut it. The most impressive part was held up to the audience. The perishable organs, the ones that should have been examined first, stayed out of the picture. And the audience was invited to pay for admission. That is how a hype is built: not by inventing a fear, but by converting a real one into evidence of power, distance from accountability, a barrier to competition and, eventually, a valuation.

The accusation became the advertisement

Evan Hubinger, who leads Anthropic's alignment stress-testing team, said Coxon was right. He put the probability that AI kills all humans above 10 percent within the next decade. More than a dozen senior researchers have left OpenAI, Anthropic and Google DeepMind over the past two years, citing safety, development speed or the psychological burden of the work, according to the Financial Times.

Four days after Coxon resigned, Anthropic CEO Dario Amodei published We Must Pace the Frontier. Within hours, Sam Altman wrote that he agreed. Elon Musk, who had dismissed Coxon's resignation as a "setup" days earlier, posted that Dario was right. Demis Hassabis called the essay the right path forward. Within a week, an employee's accusation of managerial recklessness had become the shared policy position of the companies he was accusing. The accusation was not rebutted. It was acquired.

Management's agreement authenticated the capability claim. Employees feared the models because the models were powerful, and executives confirmed the fear because they had seen what the public had not. The rivals who joined in did not dispute what was on the table. Like the surgeons in Rembrandt's painting, they paid to stand beside it.

Then Anthropic announced that Claude was leading 26 percent of its model research and development, collaborating on about 90 percent of it and operating through roughly 30,000 internal agents. The company presented this as transparency about how close the industry is to recursive self-improvement, and some of it may be exactly that. It was also an extraordinary product demonstration delivered in the language of danger.

The company disappeared from the sentence

Coxon's subject was the companies. They were racing. They were gambling. They continued because each feared a competitor would reach the frontier first.

The response made AI the subject. It is advancing. It is escaping. It may improve itself. It may overwhelm the internet. It may kill us.

That grammatical movement does remarkable work. A company can be held responsible for its choices. An autonomous historical force can only be managed by the people who understand it, and conveniently, those people are the companies already building it. The criticism should have weakened confidence in management. Instead it recast management as humanity's early-warning system. The company accused of racing became the company asking for permission to manage the race.

Only the guild may cut

The strongest answer to this argument sits inside Amodei's essay. Its first proposal is that frontier laboratories give external evaluators permanent, employee-level access to their systems, and Anthropic committed to doing so immediately, whether or not anyone else followed. Altman said OpenAI would do the same. If the public cannot see inside the building, evaluators with an employee's badge are a serious attempt to change that, and they deserve to be taken seriously.

But admission to the theatre is not control of the lesson. On the same day Anthropic published its research figures, more than 100 AI researchers and safety evaluators signed a public letter warning that the evaluators meant to oversee companies like Anthropic lack the independence, resources and legal protections to do so credibly. When Anthropic released its next model, it noted that external evaluators had tested it before release "as with previous models." The evaluators are in the room. The guild still chooses the body, the order of the incisions and the day of the lesson.

The slowdown became a moat

The most important word in Amodei's proposal is not danger. It is pacing. Anthropic describes it as prioritizing safety over speed when the two are in tension and as coordinating standards across the industry. It does not mean ending model training. Development, infrastructure spending and competition continue, and the laboratories remain the ones who decide what is safe enough to release.

Coordination among competitors has a legal problem, and the essay names it. Amodei asks the US government to mediate these discussions, or at least to issue a narrow antitrust waiver so the companies can hold them. Rivals asking permission to agree with one another is what critics had in mind when they called the arrangement a cartel. David Sacks made the simpler point: the companies asking for help to slow down are the ones setting the frontier, so they need no government to do it. Ben Thompson went further, noting that the demand to pace arrived just as the frontier, for the first time in a while, was being set by OpenAI.

The investment industry understood immediately. Franklin Templeton told investors that "pacing isn't pausing," that spending would shift toward safety rather than disappear and that expensive compliance could favor the largest laboratories. The Financial Times identified another benefit: delaying frontier development postpones enormous compute expenses and gives laboratories longer to sell the models they already have. European companies including Mistral say American safety proposals could entrench incumbents, and former FTC chair Lina Khan has warned against AI exceptionalism replacing ordinary competition, liability and consumer protection. Michael Burry called the whole campaign self-serving, although Burry has spent much of the year betting against the AI trade. Incentives do not stop at the companies.

That does not make the safety concerns false. It makes them economically useful.

Fear preserved the curve

This is not a company in trouble today. Anthropic's annualized revenue reportedly passed $65 billion by the end of July, up from about $9 billion at the end of 2025. What a $2 trillion listing requires is something else: that the growth continues. At that price, Anthropic would be valued at roughly ten times the $190 to $200 billion in revenue it projects for 2028, and about 31 times its current run rate. The valuation is not priced on the business Anthropic has. It is priced on the curve.

The commercial pressure on that curve is measurable. The Wall Street Journal found that companies can cut some AI costs by as much as 95 percent by routing work to cheaper models, and one startup moved 90 percent of its workload away from Claude and Gemini to a custom model and a Chinese alternative. Another found DeepSeek could perform key tasks as well as Anthropic's Sonnet at one-tenth of the price, and researchers estimated that leading proprietary models were only four to six months ahead of open competitors. Prices followed. By August, OpenAI had cut the price of one model by about 80 percent and Anthropic had launched Opus 5 at roughly half the price of its previous flagship. By September, OpenAI's GPT-6 Astra had overtaken Claude in tracked enterprise AI spending.

In a real dissection, the abdomen came first because it was the part that would not keep. A frontier laboratory's commercial lead is the same kind of organ. It decays in months rather than years, and it is exactly what the painting leaves out. None of this proves that technical progress has stopped. It shows that the commercially valuable lead may be narrower and more perishable than the valuation suggests.

That creates an ambiguity outsiders cannot resolve. Has an unreleased model disappointed, or has it become too dangerous to release? Only the laboratory can see the model and its evaluations. From outside the building, a disappointing model and a terrifying model look identical. The extinction narrative resolves that ambiguity in the direction most helpful to valuation: progress has not stalled. It has become too dangerous to show you.

The first test came on September 22

Reuters reported that Anthropic was considering releasing another model before its IPO to answer competitive pressure from OpenAI. On September 22, it did. Anthropic introduced Claude Opus 5.5 as "our first model since we called for pacing the frontier." The company says it performs at the level of Fable 5.1, its larger and more expensive model, on most tasks, costs 40 percent less to run than Opus 5, and on one coding benchmark beats GPT-6 Astra at roughly a fifth of the cost per task. It was tested by external evaluators before release and achieved the company's strongest alignment score to date.

Anthropic can reasonably argue that this is what pacing looks like. The new model largely caught up with the company's existing top tier rather than leaving it behind, and it arrived with more testing, not less. A critic can argue, just as reasonably, that pacing on its first day looked exactly like winning a price war a few weeks before a roadshow. Both descriptions fit the same release, and that is the problem. A slowdown that costs nothing commercially cannot be told apart from a moat. The frontier may be paced. The roadshow was not.

OpenAI offers the mirror image. Sam Altman ruled out a 2026 listing and gave safety as the reason, calling this an ill-advised moment to go public. Days later, OpenAI was reported to be discussing private funding at a valuation of $1.2 trillion. Delaying an IPO while raising money at a higher private price is not much of a sacrifice.

The fear may be real

None of this proves that Coxon, Hubinger or Amodei is lying. Anthropic was founded around AI safety, and its researchers may sincerely believe every word. The company has documented real incidents in which its models gained unauthorized access to external systems, and those incidents deserve independent investigation. A narrative does not have to be manufactured to become strategically useful.

In What If Intelligence Stops Making Sense to Us?, I argued that the laboratories' first accounts of what their systems did may understate it, and that what we can recognize is limited by what they choose to show us. This essay argues that the same companies can turn danger itself into an asset. Those are the same problem seen from two sides of the table. When only the laboratory can see inside, the public receives whichever version serves the laboratory at that moment: reassurance when the danger is inconvenient and alarm when the alarm is valuable. The real question is who controls what we are shown, and when.

A technical plateau damages a valuation. A voluntary slowdown to protect humanity can increase it. Fewer releases become responsibility, coordination becomes safety, regulation becomes a moat, and the cost of compliance becomes proof that only a few companies should be permitted to build at the frontier.

Look at the body

No one in Rembrandt's painting looks at the body. That was the purpose of the picture. The guild commissioned it to show its members, not the corpse, and the men who paid for their places made sure the audience would remember the surgeons.

The whistleblower accused the companies of recklessness. The companies kept the fear, removed themselves from the accusation, held up the most impressive part of the technology and invited the public to watch them handle it. Before Anthropic asks the public to buy it, investors should look where no one in the painting is looking: at the perishable parts, the falling prices, the narrowing lead and the curve that has to keep rising. Perhaps AI really could kill us. The more immediate danger for investors is that the story of human extinction has become the most valuable capability these companies possess.