Pope Leo XIV is concerned about AI

On his Magnifica Humanitas

There is much to be enthusiastic about in Pope Leo XIV’s first encyclical Magnifica Humanitas, published earlier this week. Already the fact that he has identified AI as a topic worthy of being the central focus of his encyclical is something to welcome, along with his awareness of many (albeit not all) of the risks brought by the steep technological trajectory we are currently on, and the need to tread carefully. This gives hope that the Catholic Church can become a powerful contributor to a broader societal alliance against the reckless way that leading AI companies are gambling with our future. But there are, as we shall see, also some complications to this optimistic reading of the encyclical.

I will soon get to some of the highlights of what Magnifica Humanitas says about AI, but let me first mention how the Pope spends two full chapters of what seems like obligatory stuff about what the Bible teaches, about Imago Dei and human dignity, and about how his teachings align with those of his predecessors. On this last topic, a special role is played by Pope Leo XIII (in honor of whom Leo XIV at his inauguration in May 2025 chose his papal name) whose pontificate lasted 1878-1903. In 1891 Leo XIII published his encyclical Rerum Novarum, about which the current Leo says this:

With that document, my beloved predecessor gave impetus to the reflection on society, the economy and politics, which is now known as the “Social Doctrine of the Church.” When some objected that the Church should not waste energy on worldly matters, but instead focus on communicating the message of eternal life, Leo XIII responded with realism and wisdom, saying that the proclamation of the Gospel cannot overlook the concrete lives of people.

When Leo XIV goes on to discuss the intervening papacies between Leo III and himself, it is largely about how they contribute to that Social Doctrine. It all points forward to Magnifica Humanitas which continues that tradition of engaging in the great worldly matters of its time.

In Chapter 3, Leo XIV arrives finally at our present-day issues around AI. One highlight is Paragraph 107 (all the paragraphs of the encyclical are numbered) which reads as follows:

107. We cannot be satisfied with merely calling for the moralization of machines — the so-called “alignment” of AI with human values — without also having the courage to insist on a further condition: the possibility of openly discussing the ethical frameworks involved and subjecting them to shared standards of social justice. Otherwise, those who control AI will impose their own moral vision, which will become the invisible infrastructure of these systems. A more moral AI is not enough if that morality is determined by a few. What is needed is a more active political involvement that is capable of slowing things down when everything is accelerating, and of protecting the opportunities for communities still to be able to participate and ask questions.

This is good stuff! The risk of power concentration and the need for democratic and political involvement including the possibility to pull the brakes — yes, those are some of the most central issues confronting us.

The following caveats about our lack of full understanding of how the AIs work, and the fallacy of thinking that the level of tomorrow’s AI will be similar to that of today’s, is also very important and very much spot on:

98. It is appropriate to preface this discussion with two considerations. First, any statement regarding AI risks becoming quickly outdated, given the remarkable pace at which these systems are developing. Second, all of us, including those who design them, possess only a limited understanding of their actual functioning. Indeed, current AI systems are more “cultivated” than “built,” for developers do not directly design every detail, but instead create a framework within which the intelligence “grows.” As a result, fundamental scientific aspects — such as the internal representations and computational processes of these systems — remain, at present, unknown. There thus emerges an urgent need for a twofold commitment: on the one hand, a deepening of scientific research; on the other, the exercise of moral and spiritual discernment.

There is much more in the same spirit, and Leo XIV speaks clearly about the dangers (but also the efficiency) of handing over various decision making processes to AI, the importance of accountability, the responsibility that AI developers have for their design choices, the downsides of mass surveillance, and the riskiness of developing autonomous weapons systems.

Eventually he comes to the topic of AI effects on the labor market and the risk of mass unemployment, about which he elaborates at some length. About some of this I am a bit more luke-warm, because what the Pope fails to do here is to properly weigh the ills of automation against how after all it has been the engine behind the last couple of centuries’ economic growth and prosperity. While he ends up in a position similar to mine, favoring going forward with the technology more carefully than a laissez-faire policy would be expected to do (he sometimes sounds like a true Scandinavian social democrat), he could have done more to address the very real tensions we are facing regarding job security versus AI-driven growth. The bigger question of whether wage labor really is the thing that society should be organized around, even when we have technology that can liberate us from that, is never properly addressed: while Leo’s answer is clearly “yes”, it is arrived at in a way that, despite all the talk about human dignity, I find oversimplified and superficial.

That is a shortcoming of Magnifica Humanitas that I would readily forgive in the interest of forming alliances against big tech and reckless accelerationists. But there is a worse flaw in the encyclical, namely the absence of any discussion at all of the existential risk to humanity that might arise with the emergence of a misaligned superintelligent AI. Although that omission is unfortunately far from unique in contemporary AI ethics literature, it is nonetheless bizarre in such an otherwise wide-ranging document on the risks arising from increasingly advanced AI. Here I will join Zvi Mowshowitz and Shakeel Hashim in critiquing this omission, which sends the implicit message that the risk of an AI apocalypse wiping out Homo sapiens from the planet is too remote or speculative to merit serious attention.

So why in the world would Pope Leo XIV have such a view? I believe the best clue to this can be found in this paragraph:

99. It is not possible to provide a single, comprehensive definition of AI. What can be stated, however, is that we must avoid the misconception of equating this type of “intelligence” with that of human beings. These systems merely imitate certain functions of human intelligence. In doing so, they often surpass human intelligence in speed and computational capacity, offering tangible benefits across many fields. Yet this power remains entirely tied to data processing. So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom. Even when these tools are described as capable of “learning,” their way of doing so is different from that of a human person. It is not the experience of those who allow themselves to be shaped by life and grow over time through choices, mistakes, forgiveness and fidelity. Rather, it is a form of statistical adaptation based on data and feedback, which can be very effective, but does not imply inner growth.

This insistence that AI is incapable of possessing (rather than merely imitating) real intelligence is clearly in some tension with the above-quoted Paragraph 98, although perhaps not quite a full contradiction.¹ From such a position of intelligence denial it may not be such a huge conceptual leap to assume that AI can never reach the level of capabilities where it can truly challenge humanity’s place at the top of the earthly hierarchy. The intelligence denial position is essentially the same as that held by Bender-Gebru-style AI ethicists in the academic left, and the conclusion that existential risk from AI is not a concern is the same. My own view, needless to say, is that the position is mistaken, but there is a way in which I respect it more coming from the Pope (or other religious quarters) than from the Bender-Gebru camp. From the latter, the arguments for the position (to the extent arguments are presented at all) tend to be along the lines of “just predicting the next word”, “just stochastic parrots”, or “just matrix multiplication” — all instances of the reductio ad reductem fallacy: the idea that if a system can be shown to be made of very simple constituents, then it cannot possess intelligence or other interesting emergent properties. The final sentence of Paragraph 99 (“a form of statistical adaptation”) hints in the same direction. The standard reply to attempts at reductio ad reductem is to point to the obvious counterexample: the human brain consists merely of atoms and elementary particles mindlessly bumping into each other, and yet intelligence emerges. Unlike the likes of Bender and Gebru, Pope Leo XIV can claim that the counterexample is invalid because the reductionist and physicalist view of the human mind is incomplete: in addition to the material components we possess immortal souls bestowed by God. I believe he would be mistaken here, but at least it is a coherent worldview.

Let me end with what might be the juiciest observation about Magnifica Humanitas, namely that large parts of it are (very likely) AI-generated. The evidence for this is overwhelming, as outlined in separate short reports by Daniel Filan and by Linch, the latter also pointing out Claude as the most likely AI assistant involved in the writing. What exactly to make of this is unclear. On one hand, I am not generally against using AI tools for polishing one’s drafts or for providing input to the writing process in other ways, and in fact I often consult AI in my own writing. On the other hand, there is something icky in allowing AI to influence the content of documents that are likely to impact humanity’s overall strategy for our transition to a world with advanced AI. This is especially so if one takes seriously (as I do) the AI alignment problem and thus the possibility of misalignment and a conflict between AI’s values and ours.

1. It is worth noting that the otherwise extremely polite address made by Anthropic co-founder and AI safety researcher Chris Olah at the official presentation of Magnifica Humanitas on May 25 hints at the wrongness of Paragraph 99.