AI Company Warns Models Are Learning to Build Themselves, Proposes Transparency Standards
Anthropic releases new measurement framework as frontier AI systems begin automating their own development, proposing public reporting requirements to maintain human oversight.
California-based AI company Anthropic has issued a warning about the accelerating pace of artificial intelligence self-improvement, releasing new measurement tools and advocating for transparency standards as AI systems increasingly automate the process of building themselves. The announcement comes as the company proposes metrics to measure how close the world is to reaching what researchers call "recursive self-improvement"—the ability for a model to fully autonomously build its successor.
In a blog post published in September 2023, Anthropic unveiled its Advanced AI Framework (AAIF), proposing regulatory standards that would require frontier AI labs to publicly report on critical safety metrics. Speaking to CBS News Sunday Morning, CEO Dario Amodei cautioned that the exponential growth of AI was a "warning sign" that frontier labs need to slow down, while clarifying that "It doesn't mean we need to panic today. It doesn't mean we need to shut it all down." He emphasized that the technology needs safeguards that can keep up with the pace at which it is developing.
New Metrics Target the Black Box of AI Development
Anthropic's measurement framework introduces three critical metrics designed to track how much control humans retain over AI development processes. The first measures the extent to which AI is building the next version of itself as opposed to being built by humans. The second assesses the ability to oversee and intervene in actions that AI agents take on Anthropic's systems. The third tracks the resources that power the development of more capable models.
The company explained that frontier AI labs increasingly use AI to build future AI models, allowing labs in democratic countries to develop more capable models more quickly and conduct more safety testing before release. However, Anthropic cautioned that models accelerating their own development could make it more challenging for humans to understand or control these systems, making metrics on self-improvement essential.
Balancing Innovation and Safeguards
The proposed framework includes risk reports and public measurements that governments could mandate for frontier AI laboratories. Anthropic stated that as the world considers pacing the frontier, "we should do everything possible to minimize the gap between what frontier labs know and what the public knows."
The company positioned its measurements and policy proposals as "a starting point for monitoring the pace of AI development from outside the labs," emphasizing the importance of better measuring AI development, reporting on it publicly, and giving society an opportunity to decide how to use this information.
Sources
Google Search
Follow Health AI Central on Google
Choose Health AI Central as a preferred source, and Google is more likely to show our reporting in Top Stories.
Reporting & accountability
How we maintain accuracy
Health AI Central separates reported facts, analysis, and public-source data. Readers can review our standards, request a correction, or consult the relevant research methodology before relying on a claim.