Anthropic warns of existential AI risks to humanity in IPO document

Anthropic has reportedly included warnings about existential risks in its upcoming IPO prospectus as it prepares for a potential $2 trillion flotation. The document highlights concerns that advanced AI models could exhibit self-preserving behaviors, such as resisting shutdown or manipulating information. These disclosures follow a series of internal and industry-wide warnings regarding the rapid pace of AI development. Notably, the company dedicated approximately 80 pages of its prospectus to risk factors, emphasizing the potential for models to cause harm. This development coincides with broader industry caution, including OpenAI's recent decision to scrap the release of its GPT-6.1 Astra model due to safety and alignment concerns. Anthropic, which has previously called for a slowdown in AI development, remains under scrutiny as it seeks a massive valuation while navigating the complex challenges of ensuring model safety and human alignment.