Anthropic Cofounder Chris Olah's Remarks on Pope Leo XIV's "Magnifica Humanitas"

geerlingguy

> And what has grown is far more subtle, odd, and beautiful than science fiction prepared us for. They are not the cold, calculating robots we were promised. They are made from us, from our words—and, as the Holy Father observes, they remain in important ways mysterious even to those of us who train them I love how he's framing AI as some new and fascinating form of consciousness... when in fact it is a cold, calculating technology devoid of any empathy or care.

RobotToaster

Thou shalt not make a machine in the likeness of a human mind

only_in_america

I don't know about you but his remarks read like AI. I don't think he was taking it seriously.

malux85

AI is a golem

bix6

So flat compared to the pope’s work. And puts all the impetus on the church instead of taking responsibility.

zeafoamrun

Whole lotta em-dashes in that speech.

A_D_E_P_T

> The first is our duty to the global poor. There is a real possibility that AI will displace human labor at very large scale. If that happens, supporting those displaced will be a moral imperative of historic proportions. This guy doesn't understand what the global poor actually do for a living. They're not lawyers or paper-pushers, nor do they work in medical diagnostics. They're usually farmers. Sometimes they work in craft businesses, in fishing boats, or in various mercantile trades. Nobody's even talking about how AI is going to displace that kind of labor, because it's hard to do, hard even to conceive, and it doesn't seem likely to happen in the near term. Lawyers and judges can already be automated, but a yeoman farmer?

zugi

LLMs are software. They take inputs and produce outputs. What humans choose to do with those inputs and outputs is up to us. Getting the pope involved makes it all seem more mystical and magical than it is. And these remarks only further feed that delusion. Regardless of intent, it seems to just feed the AI marketing and hype.

embedding-shape

> AI development is concentrated in a handful of wealthy nations. How can we ensure the gains of AI are shared globally? We do not have a mechanism for this. It is an unsolved problem Kind of ironic given almost every AI lab except the one you started and work for actually done model releases to the public, some more "open" than others, but still something. Look around at what other companies are doing, Qwen/Alibaba seems to have found a pragmatic middleground where they keep the most powerful model variant closed source and only API-accessible, while other models are being released openly to the public, to the entire world in fact, and when the next model release comes around, the previously undisclosed model has now been superseded. I wonder if Chris ever copy-pasted his writing into Claude and asked something like "Please review this honestly and give me raw feedback, and challenge every claim that is weak", seems there are more "not really reflective of reality" points than just the above.

imjonse

"How can we ensure the gains of AI are shared globally? We do not have a mechanism for this" Somebody inform Anthropic about open models and research.

SilverElfin

I don’t mind the actual content here. But I do find it disturbing that a company soon to before a monopolistic force affecting all our lives may have deep ties or be influenced by one religion or the other. This isn’t the first such tie up with organized religion either. Anthropic and OpenAI also have done work with the interfaith alliance. See their “Faith - AI Covenant”: https://iafsc.org/our-work/faith-ai-covenant I hope we don’t see safetyism, which is already a problem (see age verification and social media moderation), evolve into some sort of religious moralization implemented through AI providers.

sorokod

Chris Olah and other leaders at Anthropic, OpenAI, and others would do well to consider the principles of Social Doctrine spelled out in the encyclical. The question they should ask themselves is how their corporations advance those principles. Olah argues that "if we want this technology to go well, it is enormously important that there be people outside those incentives." That sounds part hypocritical and part evasive; the responsibility starts with the people inside the incentives — with him.

_pdp_

> They are not the cold, calculating robots we were promised. I am not sure how anyone even remotely familiar with how LLMs work can say this. This is a fine-tune job. > The first is our duty to the global poor. I don't think they are affected by AI as much as low and middle class but I am not economist. > How can we ensure the gains of AI are shared globally? Opensource? > We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. Such an Anthropic thing to say. LLMs experience joy and grief? > We need informed critics who will tell the labs when we are failing I don't think anyone is as informed as they think they are. Obviously nobody has been through this before so it is safe to assume that even experts are dead wrong.

catigula

> The third is the need for discernment on the nature of AI models. I am a scientist. I lead a research team that studies the internal structure of these models—what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment Very interesting because it feels like the rudiments of an “AI rights” argument. If we can produce artificial minds with rights and dignity, there is no need for humans, and their voices will quickly drown out to obscurity. It is a fairly obvious doomsday scenario.

kwanbix

> The first is our duty to the global poor. There is a real possibility that AI will displace human labor at very large scale. If that happens, supporting those displaced will be a moral imperative of historic proportions. We already have poors that are really suffering, and the "elite" (or oligarchs depending on your point of view) have done very little to help them. Why should we trust them they will do anything for us if we are all displaced by AI?

yshamrei

Will there be a battle? =) I think Chris Olah is obviously a huge enthusiast of his work and sincerely believes that what he is building will benefit humanity. At the same time, he is influenced by his environment and goals. He dreams that new technologies will invent something that significantly improves the lives of people who do not even care about these technologies. I also think Pope Leo XIVs probably does not deeply understand new technologies and AI in general. But his role is to be cautious about anything that could potentially be used against humanity’s interests. And honestly, despite the good intentions of inventors, nobody can predict how humanity will ultimately use these technologies. AI is already using in wars. And in general, the Church has historically been cautious about progress in almost any form. What definitely unites both Chris Olah and Pope Leo XIV is faith. Faith in their goals and ideals.

bachmeier

"There is a real possibility that AI will displace human labor at very large scale." "If AI models are going to be widespread" "we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease." A critic might charge that this is nothing more than "let's keep the AI hype rolling so the money keeps coming in". Surely the promotional statements (and the third one, which is marketing nonsense) were not necessary if they actually cared about the issues they're claiming to care about.

rzmmm

It's like reading the Mythos preview card. He talks like their AI is some sci-fi monster. Curl developer put it well: "Mythos was mostly marketing"

dang

Related ongoing thread: Magnifica Humanitas - https://news.ycombinator.com/item?id=48265206 - May 2026 (493 comments)

tilltheend

Damage control, double speak, false agreement to misinterpret Leo XIV words.