The situation we are in
A stark message from Beth Barnes
Beth Barnes is founder and CEO of the AI safety organization METR (Model Evaluation and Threat Research), which works in close collaboration with leading AI developers including Anthropic, OpenAI and Google DeepMind. In March last year, they published their report Measuring AI Ability to Complete Long Tasks, containing a graph that I have shown, along with updated versions, in 20+ of my talks since then (such as this one, and this one) and called it “the most important diagram published in 2025”.1
Barnes has been outspoken about AI risk, such as in conversation with Rob Wiblin last summer. Earlier this week, she stepped up her messaging with the following concise and stark statement, which is pretty much in line with what I have been saying with increasing urgency for the last several years. Coming from her, however, it carries so much more weight, due to her almost unique position of direct insight into what is happening at the epicenter of AI development:
Sometimes people outside the field say things like “The AI situation can’t be that bad, there must be experts who are on top of it”. As “an expert”, I would like to be clear that we are *not* on top of it. Some key aspects of the situation IMO:
(1) We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years.
(2) Things are chaotic and rushed; we aren’t on top of the basics (models regularly violate user intent, labs train on things they meant to avoid, security probably isn’t good enough to prevent adversaries stealing dangerous models) let alone thorny questions of how to control/align superhuman AI.
(3) METR (and other independent orgs, as well as safety/security teams at labs) feel woefully under-resourced compared to the scale and pace of AI development - we’re struggling to build benchmarks fast enough, keep ahead of latest capability developments, read and respond to all the safety-related claims that AI developers are making, run all the evaluations and assessments that companies + governments are asking us to, plus develop the science needed to assess risks from increasingly capable AIs.
(4) IMO, any “reasonable” civilization would clearly be taking things much more slowly and carefully with AI. The benefits of getting upsides of advanced AI a little faster are small compared to the risks of getting it irrecoverably wrong, and we could lower these risks by going slower.
We should listen to Beth Barnes. And we need to get our act together and alter this crazy trajectory we are on. Politely asking industry leaders like Dario Amodei and Sam Altman to coordinate on going slower and with greater care is not going to suffice, because they seem at present to be stuck in the dangerous and self-fulfilling idea that such coordination is impossible. What we instead need is legislation and binding international agreements. Many ideas for how that might look have been proposed, such as this one from MIRI (Machine Intelligence Research Institute). We have no time to waste.
Their more recent Frontier Assessment Report is also alarming and important.