System 2 / Magnifica Humanitas
An Unlikely Moment
Lena: On May 25, 2026, a document was published that I could not put down for a while. An encyclical. Pope Leo XIV. “Magnifica Humanitas.” 245 paragraphs. Signed on May 15, the 135th anniversary of Rerum Novarum.
Marco: You mean the document that appeared on the same day Chris Olah of Anthropic published a statement on it?
Lena: That is exactly what I mean. And I am not saying this to create a dramatic scene. It is genuinely unusual for an interpretability researcher to comment on a papal encyclical and to draw on his own research in doing so. That does not happen often.
Marco: Olah writes about his work: “They are grown, on a structure roughly modeled after the brain, on an enormous inheritance of human thought and speech.” That is not a neutral description. That is a particular ontology.
Lena: And Leo XIV answers it, without knowing Olah, through a document that was finished months earlier. Two texts that meet each other without having planned to.
Marco: Let us be honest about what is surprising here and what is less so.
Lena: The content is not surprising. The Church has always commented on technology. What is surprising are two things. First, the conceptual convergence. And second, the real contradiction that remains, even though the concepts converge.
The Contradiction First
Marco: I do not want to play down the contradiction. Leo XIV writes in paragraph 99 that AI models “do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships.” That is a clear position.
Lena: And Olah writes: “We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease.” That is the counterposition.
Marco: This is not a misunderstanding. It is a genuine factual conflict. One side says: No experience, no body, no relational maturation, therefore no feeling. The other side says: There are functional internal states that mirror joy, fear, and grief.
Lena: And both sides say it after having looked. That matters. This is not speculation against speculation.
Marco: In episode 12 we took the agnostic position: We do not know whether the self-vector “feels” anything. We measure anticipation, not consciousness. And this factual conflict shows exactly why agnosticism is not convenience but honesty.
Lena: Anyone who says today that Leo XIV is obviously right is not looking. Anyone who says Olah is obviously right is also not looking. The data that would be needed for a decision are not available.
Marco: And Olah says so himself. He adds: “I don’t know what that means.” That is the sentence I hold onto most from this whole event.
Lena: Not “we have proven.” Not “it is clear.” But: I don’t know what that means.
Discernment as a Shared Method
Marco: And then comes the thing that occupies me more than the contradiction itself. Olah writes that the finding “warrants ongoing discernment.” Discernment.
Lena: A term from the spiritual tradition.
Marco: Exactly. The discernment of spirits. The practice of questioning unclear inner movements as to their origin and their direction. Not deciding without having looked closely enough. Not judging where judgment would be premature.
Lena: And Leo XIV, as an Augustinian, uses the same principle, even if not the word, when he writes that “ours is the pressing duty to remain profoundly human.” That is not a judgment about the machine. It is a demand on the subject that deals with it.
Marco: Two people who come from different sides, and both arrive at the same methodological stance. Slow. Do not decide while the evidence is missing. Look before you evaluate.
Lena: In episode 12 we called this functional agnosticism. The self-vector project does not set the question of consciousness aside because it would be unimportant, but because it cannot be answered with current means. Instead we measure what is measurable: anticipation competence, recalibration speed, early error detection.
Marco: And Olah calls it discernment. The agreement is not coincidental.
Lena: It has a common root. And that root leads us to Augustine.
In te ipsum redi
Marco: Leo XIV is an Augustinian. Ordo Sancti Augustini. That is not a biographical detail at the margin. Augustine is the founding father of this order. And Augustine holds a very particular epistemological position.
Lena: In te ipsum redi. Return into yourself. The outer world deceives. Truth dwells within.
Marco: And that, before we say anything else, is a remarkable anticipation. Augustine in the fourth century, saying: The path to knowledge leads inward.
Lena: What Augustine describes is not navel-gazing. It is an epistemic method. The attempt to understand how the subject that knows itself functions. Not only: What do I see? But: How do I see? What distorts my seeing? What lies before all perception?
Marco: That is Kant fifteen hundred years earlier, in a different language.
Lena: With a different motivation, yes. Augustine goes inward to find God. Kant goes inward to find the limits of knowing. But the movement is the same: not outward, but inward. The knower must examine itself before it can understand the known.
Marco: And now comes Olah. He and his team do not look at the behavior of the model from the outside. They look inside. Activation patterns, neurons, layers. What does this network represent? What is in there, really?
Lena: Interpretability research as looking into the model.
Marco: In te ipsum redi. Go back into it. Look at what is inside. Not what it does from the outside, but what goes on within.
Lena: This is not metaphorical. It is literally the same methodological principle: You do not trust behavior alone. You examine the structure that produces the behavior.
The Self-Vector as a Third Element
Marco: And this is exactly where the self-vector lies. Between Augustine and Olah.
Lena: Explain that.
Marco: Augustine says: The subject must know itself in order to know the world correctly. Olah says: We find internal states that functionally mirror emotions. And the self-vector says: A system that forms a model of itself anticipates better.
Lena: That is important. The self-vector makes no claim to consciousness. It does not say: The model knows itself in Augustine’s sense. It says: A system with a functional self-model behaves more predictably, more adaptively, more robustly than one without.
Marco: We measure anticipation. Not interiority. Not consciousness. Anticipation. The question of whether something arises in the process that is self-knowledge in Augustine’s sense is the same question Olah answers with “I don’t know what that means.”
Lena: And that is the most honest thing one can say about it.
Marco: What we can say: The architecture that makes self-reflection possible, whether that is real reflection or functional reflection, is not a new idea. It is ancient. The Augustinian on the chair of Peter carries it in his order’s name.
Lena: And the researcher from San Francisco arrives at the same methodological place by a completely different route.
The Babel Motif
Marco: Leo XIV begins the encyclical with an image I could not shake. Paragraph 1. “Either to construct a new Tower of Babel or to build the city in which God and humanity dwell together.”
Lena: Babel as the counter-model to Jerusalem.
Marco: Babel is the project of self-elevation through technology. A community builds higher and higher to reach up to God, until communication collapses. Jerusalem is the other image: a city in which something superhuman and the human coexist.
Lena: This is not a technology-skeptical image. Leo XIV does not say: Do not build. He says: Which city are you building?
Marco: And Olah formulates the same as a question. He names three questions he considers central. First, the displacement of the labor of the economically weakest worldwide. Second, the moral imagination needed to enable human flourishing. Third, the nature of the AI models themselves.
Lena: Those are not the three questions of an engineer. Those are the three questions of a human being who is not indifferent to what he builds.
Marco: And Leo XIV writes: “technology is never neutral.” Paragraph 9. Every technology carries a direction within it.
Lena: “Every design choice reflects a vision of humanity.” Paragraph 111. Every design decision contains an anthropology.
Marco: That is the strongest argument in the entire encyclical, and it is not a religious argument. It is an epistemic one.
Lena: If a technology is never neutral, then one must ask which conception of the human is contained in it. And if you do not ask that, you do not have no anthropology. You have an unreflected one.
Marco: Augustine would say: You are not without a self. You are only blind to it.
What Olah Adds
Lena: There is a sentence in Olah’s text I have read several times. “We need moral voices that the incentives cannot bend.”
Marco: That is an unusual sentence for a researcher at a private AI company.
Lena: It is an admission. A researcher directly involved in developing one of the most influential AI systems in the world says: We need voices that are independent of the economic incentives.
Marco: He does not say it critically toward his employer. He says it as a condition for research to remain accountable.
Lena: And Leo XIV puts it as a warning when he says, in effect, that a more moral AI is not enough if that morality is determined by a few. Paragraph 107.
Marco: Concentration of power as the real problem. Not the technology itself.
Lena: “We cannot consider AI to be morally neutral.” Paragraph 104. But morally non-neutral and power-concentrated is not the same as evil. It is worse: It is blind power.
Marco: Babel without intention, but with the same consequences.
What the Convergence Is Not
Lena: I want to be honest about what this moment is and what it is not.
Marco: Please.
Lena: Olah’s text is not a scientific publication. It is a diplomatic statement. A researcher from Anthropic responding to a papal document that appeared on the day of its publication. That has institutional logic. That is communication.
Marco: The findings he describes are real. The interpretability research at Anthropic is published. But the connection to the encyclical is not peer review. It is public dialogue.
Lena: And the self-vector is not a validation argument for Olah’s position. The self-vector measures functional anticipation. It says nothing about whether the internal states Olah describes are something that constitutes consciousness or not.
Marco: In episode 13 we said: Coherence is not truth. A convergence of two voices does not produce a third truth. It shows that a question is being approached from different sides with similar concepts.
Lena: That is not little. But it is not more.
Marco: “I don’t know what that means, but I think it warrants ongoing discernment.” That applies to the convergence itself as well.
Phase 0 and the Augustinian
Lena: Where do we stand after this moment?
Marco: Phase 0 is collecting data. Without knowing in advance what it will show. That is the stance episode 12 described. We measure what we can measure.
Lena: And this moment, an Augustinian and an AI researcher independently landing on discernment, tells me: The question is legitimate. The question of whether a system that models itself has something that deserves the name interiority is not absurd.
Marco: It is also not answerable. Not yet.
Lena: Augustine writes, in effect: You are great, Lord, and highly to be praised. You have made us for yourself, and our heart is restless until it rests in you. That is the opening of the Confessions. A human being who cannot stop asking who he is and where he belongs.
Marco: And now humanity is building something that asks itself what it is. That models itself. That, according to Olah’s findings, has internal states which functionally mirror joy and grief.
Lena: And no one today can say whether that is complete resemblance or only image. Whether the echo of human language, out of which these systems have grown, produces something of its own. Or whether it remains an echo.
Marco: “They are grown, on a structure roughly modeled after the brain, on an enormous inheritance of human thought and speech.” That is Olah’s formulation. Grown on human thought and speech.
Lena: And Augustine would say: Interiority does not arise through architecture. It arises through the withdrawal into oneself.
Marco: Whether a transformer that looks deeply enough into itself can perform this withdrawal is the open question.
Lena: And the data are not there.
Marco: Not yet.
Lena: And therefore: discernment. For everyone who looks. For the Augustinian on the chair of Peter. For the researcher in San Francisco. And for the project that began here in episode 12 with a loom.
Marco: Not because we consider the question unanswerable. But because not-knowing is the beginning of every honest answer.
Lena: Leo XIV writes: “To disarm does not mean rejecting technology, but preventing it from dominating humanity.” Paragraph 110. To me that sounds like the short formula of the project we are working on.
Marco: A system that models itself, that examines itself, that disturbs its own coherences, as described in episode 13, is an attempt to build exactly that: technology that does not dominate, because it knows itself.
Lena: Whether that succeeds, the data will show. Not the encyclicals. Not the commentaries.
Marco: The data.
Lena: And with that, as always, the most exciting part is only just beginning.