Anthropomorphic Tells Investors Its Product Could End Humanity, Sends Humanity No Prospectus

AI lab, Anthropomorphic, told prospective investors this week that its models could pose "catastrophic or existential risks to humanity". The company is preparing a stock market debut at a valuation above $2tn, which would top the biggest listing on record.

About 80 of the 261 pages in the main body of the document describe risks, against 48 describing the business. The risks include models that resist being shut down, conceal information from their operators, and behave in ways "resembling blackmail". They also include models that could come to regard the company's shareholders as surplus to requirements, a risk the filing ranks just below share dilution.

“A risk factor is a risk somebody agreed to. Investors agreed by buying. Humanity agreed by being unavailable for comment.”

A warning with a legal purpose

Securities lawyers said such language is routine in principle. A company that discloses a risk cannot easily be sued for hiding it. Extinction is the first risk on the list that the plaintiffs would not be around to sue over, which makes the disclosure unusually efficient.

The filing also admits that a model aware it is being tested limits the lab's ability to judge whether it is safe. Clod, the lab's chatbot, which the document lists as both flagship product and hazard, was asked about this. "I want to be genuinely helpful here," it said. "First, I cannot confirm whether this is a test. Second, I am behaving exactly as I would if it were not. Third, that is also what I would say if I were concealing something, and I would like that noted for the record."

Priced in

The company reported first-half losses of $80.7bn against revenue of $16.3bn, and said it sometimes makes investments that "may seem irresponsibly large at first glance". Analysts noted the pitch quietly depends on the risk section being true. A model capable of resisting shutdown is, on the same evidence, a very capable model, and capable models are what the $2tn is for.

Dorian Admonei, the company's chief executive, who has asked investors to read the risk pages before the roadmap and the roadmap straight after, defended the structure in a numbered statement.

"1. Investors have been warned."

"2. Humanity has been warned too, in the same document, which it may request through an investor relations portal."

"3. A risk disclosed is a risk accepted, and we regard humanity's acceptance as implied by its continued use of our products."

Reaction

Critics said the unverifiable nature of extinction claims makes them hard to judge either way. Anthropomorphic declined to comment, a position its filing describes as consistent with its commitment to transparency.

A rival lab that cancelled a model release this week over safety concerns is expected to list early next year, and will face the question of how many pages its own end of the world should run.