The fog is thick
A talk I gave last month at Stockholm AI Safety
On April 15 this year I had the pleasure of visiting Stockholm AI Safety (SAIS) and give a talk that I had chosen to call The fog is thick, the precipice may be near, and we are ill-prepared. The title is deliberately melodramatic, although not in an ironic way but simply because I think it fits the current Crunch Time for Humanity. I wanted to give the SAIS people some stuff that goes a bit beyond the usual basics on AI risk and my most oft-repeated talking points, but regular readers of this Substack are likely to recognize much of the material from earlier posts such as Superintelligence and existential AI risk ought to be taken seriously and Straight talk. There is an error near the end of the talk, at the penultimate slide, where regrettably I attribute a recent quote by Rob Bensinger to Katja Grace; my apologies to both of them. Anyhow, without further ado, here is the video recording of my talk:
Great presentation, thanks for sharing it! The outdated beliefs part was very interesting, and the way you talked about your book was hilarious!
Regarding the counter-arguments to xrisk, I believe that ot would be very helpful to put time and effort into explaining why #2 is incorrect. You mention the mathematical theory behind this, but no intuition. I believe that gaining a deeper intuitive understanding (beyond the fact that our atoms might have a better use for the AI) may help people to take AI xrisk seriously. I believe that people are looking for signs of this happening, and once they see a real risk based on those, that is when they start acting.
Another interesting part was the discussion of why aligned AI may still be problematic. Providing some more details about that could also be a reason for pausing/stopping AI development.