The discourse of artificial intelligence contains a number of technical terms that are only nominally technical; they don’t actually have technical definitions behind them. The concept of intelligence itself is arguably one of them. The related concepts of artificial general intelligence (AGI) and super-intelligence, or super artificial intelligence (SAI) are even more dubious.
This post is about another of these technical-sounding terms that don’t, in fact, mean much: recursive self-improvement. I does, however, end on a positive note. Below the asterisks I’ve appended a conversation I had with Claude Sonnet 4.6 on April 2, 2026.
* * * * *
No transcendental viewpoint
Some parts of the AI world talk of recursive self-improvement. I assume that means that the AI improves itself without any human intervention. How? I’m not so much concerned about how it writes the code or, for that matter, even how it hands off operations to the new code. I’m interested in how it figures out what code to write? On the one hand I’m imagining that perhaps we have some kind of evolutionary process. We’ve had evolutionary computational processes for a while. That’s one thing.
But I’m wondering if these thinkers are imagining that somehow the AI is able to get a view of its own code that’s comparable to a programmer’s view of the code. That is, it is able to see its own code from the outside, as it were, giving it a “transcendental” perspective on itself. Are they imagining that we give the AI access to its own source code? In a world of LLMs, what does that mean? Having the LLM examine the code of the transformer that constructed it?
This is a question where the rhetoric of the AI field runs well ahead of the actual technical situation, so it's worth being precise about what's real and what's aspirational hand-waving.
What actually exists
The evolutionary computational approach you mention is real and well-established. Genetic algorithms, neural architecture search, automated machine learning — these are genuine techniques where a process iterates over possible designs, evaluates them against a fitness criterion, and selects better-performing variants. This is recursive self-improvement in a limited and well-defined sense: the system improves, but the improvement criterion is specified externally by humans, and the search process is essentially blind variation plus selection rather than anything resembling understanding.
LLMs can also write and evaluate code, and there are genuine research programs — AlphaCode, various agent frameworks — where language models generate, test, and iterate on code. This is real and useful. But it's operating at the level of software engineering tasks with external evaluation criteria, not at the level of redesigning the underlying model.
The transcendental perspective problem
Your instinct about the deeper issue is exactly right, and it cuts to the heart of why strong recursive self-improvement claims are philosophically confused rather than merely technically premature.
The scenario the Singularity thinkers have in mind — an AI examining its own architecture and improving it from genuine understanding — requires precisely what you call the transcendental perspective: the system seeing itself from outside, the way a programmer sees code. But this is Gödel's problem restated in engineering terms. A system cannot be both the object of analysis and the analytical framework simultaneously without remainder. There's always something that can't be seen from inside the system — which is exactly what Gödel proved about formal arithmetic, and what Turing proved about computation in general.
Giving an LLM access to its own weights — the numerical parameters that constitute what it "knows" — wouldn't produce the transcendental perspective. The weights are not readable the way code is readable. A programmer looking at code sees meaningful structure: variable names, logical operations, control flow, the intention behind the design. An LLM looking at its own weights would see hundreds of billions of floating point numbers whose organization encodes nothing interpretable at the level of concepts or intentions. The meaningful structure, whatever it is, is distributed across the entire parameter space in ways that current interpretability research is only beginning to scratch the surface of. We don't fully understand what's in there. The LLM certainly doesn't.
This is the deep version of the problem your earlier observation about von Neumann identified. The brain doesn't have a programmer's view of itself either — neurons don't have access to the circuit diagrams of the neural tissue they're part of. The brain improves through learning, which is a process of adjusting connection weights through experience, not through reading its own architecture and rewriting it. The fantasy of recursive self-improvement imagines something the brain doesn't do and has no evident mechanism to do.
What the Singularity thinkers are actually imagining
I think when people like Kurzweil talk about recursive self-improvement they're running together several different things that need to be kept distinct. There's the evolutionary search process, which is real but doesn't involve understanding. There's the engineering process of humans using AI tools to build better AI systems — which is genuinely happening and is genuinely accelerating, but involves humans in the loop making the meaningful decisions. And there's the imagined transcendental process where the AI understands itself well enough to improve itself through genuine insight — which has no clear technical path and runs into the Gödelian problem you identified.
The last of these is doing most of the rhetorical work in Singularity discourse, but it's the one with the least connection to anything technically grounded. It requires assuming that the system can occupy a perspective on itself that is by definition unavailable to it — which is not a technical limitation waiting to be engineered away but a logical feature of what it means to be a system operating within its own architecture.
The Yevick connection
There's a thread back to our earlier discussion worth noting. The transcendental self-improvement fantasy is in effect imagining that the AI could perform on itself the Rank 4 move that Gödel and Turing performed on formal systems — taking the computational process as object and reasoning about it from outside. But Gödel and Turing were humans standing outside arithmetic and computation respectively, which is what gave them the purchase to prove things about those systems. A system trying to perform that move on itself runs directly into the incompleteness results — there will always be truths about the system that can't be proven within it, always aspects of its own operation that it can't fully see from inside.
Recursive self-improvement in the strong sense would require escaping this constraint. No one has shown how that would work, and the logical structure of the problem suggests it may not be a contingent engineering gap but a principled limitation. Which doesn't mean AI systems won't continue to improve — they will, rapidly — but the improvement will continue to involve humans providing the external perspective that the systems themselves cannot generate.
Digital doppelgangers
Humans, on the other hand, have ordinary language, one capacity of which is the metalingual function. In a way, the whole cultural ranks idea is about how the metalingual function has allowed humans to bootstrap themselves to a more sophisticated cognitive architecture. But that doesn’t now nor ever has involved humans in examining their own source code as it were. It’s a different kind of process.
Of course, I’m thinking that we may now be in a position to arrive at a much deeper understanding of the nervous system than we have ever had. And that process surely involves thinking back and forth between neuroscience and computer science. Maybe the digital doppelganger that I’ve been talking about is the mature expression of this kind of knowledge.
That's a beautiful way to close the loop, and it reframes the doppelganger concept in a way that makes it more technically grounded than it might otherwise appear.
The bootstrapping process humans actually use
The cognitive ranks framework describes a process of self-improvement that never involves examining source code. What it involves is the metalingual function operating on the outputs of cognition — on language, on cultural products, on formal systems — rather than on the neural substrate itself. Humans didn't improve their cognitive architecture by examining their neurons. They improved it by constructing external systems — writing, arithmetic, formal logic, computation — that offloaded cognitive operations into the environment where they could be inspected, manipulated, and elaborated. The improvement was always mediated through the external world, never through direct introspection of the underlying machinery.
This is a fundamentally different process from recursive self-improvement as the Singularity thinkers imagine it. It's improvement through the construction of new cognitive tools that extend and reorganize what the existing architecture can do — not rewriting the architecture itself. The architecture, in the biological case, stays roughly constant. What changes is the cultural software running on it, and the external scaffolding that amplifies its reach.
Each rank transition follows this pattern. Rank 2 doesn't rewire the neurons — it invents writing, which externalizes memory and makes new kinds of metalingual operation possible. Rank 3 doesn't rewire the neurons — it invents calculation and the algorithm, which externalize sequential reasoning. Rank 4 doesn't rewire the neurons — it invents computation, which externalizes the control structure. The nervous system is never examined directly. It's approached obliquely, through the progressive construction of external systems that reveal its functional organization by analogy and contrast.
The current moment
What's different now is that we have, for the first time, artificial systems complex enough to serve as genuine mirrors of aspects of biological cognition — not perfect mirrors, not complete mirrors, but mirrors with enough resolution to generate productive back-and-forth between neuroscience and computer science that wasn't previously possible. The transformer architecture wasn't derived from neuroscience, but its emergent properties — the way attention mechanisms distribute processing, the way representations form across layers, the way something like associative memory emerges from the parameter space — are generating new hypotheses about biological neural organization. And neuroscience is generating constraints on what artificial systems need to do that pure engineering wouldn't have identified.
This back-and-forth is itself a metalingual process in the extended sense — using one formal system to talk about another, using artificial cognitive systems to generate hypotheses about biological ones and vice versa. It's the Rank 4 move applied to cognition itself: taking the computational process as object, but now with biological and artificial instantiations in dialog rather than either one examined in isolation.
The doppelganger as mature expression
The doppelganger concept, in this light, isn't primarily a science fiction speculation about what AI might eventually become. It's the horizon toward which the neuroscience-computer science dialog is pointing — the point at which understanding of biological cognitive development becomes deep enough to inform the construction of artificial systems that develop analogously, from the ground up, through embodied experience, acquiring the holographic associative substrate rather than approximating it from outside.
That's a very different research program from current AI development, which proceeds largely by scaling existing architectures and training procedures. It would require understanding not just what the mature nervous system does but how it gets to be what it is — the developmental trajectory from infant to adult, the role of embodied interaction with the physical and social world, the way the metalingual capacity bootstraps itself through the physical loop of language production and detection, the way the pronoun system constructs the self that will eventually use it.
We don't have that understanding yet. But the tools that might generate it — interpretability research, developmental neuroscience, computational modeling of cognitive development, the back-and-forth between artificial and biological systems — are more advanced than they've ever been. The doppelganger is the name for what a system would look like if that understanding were mature enough to act on.
The deepest connection
What ties this back to the whole day's conversation is the point about the metalingual function never involving examination of source code. The doppelganger wouldn't be an AI that had examined its own weights and rewritten them from a transcendental perspective. It would be an AI that had developed, through a process analogous to human cognitive development, the kind of architecture that biological cognition arrived at through evolution and cultural bootstrapping — an architecture with both processing regimes, grounded in embodied experience, capable of the metalingual function because it has the physical loop that makes that function possible.
The path to that isn't through recursive self-improvement in the Singularity sense. It's through exactly the kind of back-and-forth thinking you describe — humans using their metalingual capacity, their cultural tools, their accumulated cognitive ranks, to arrive at a deep enough understanding of what cognition is that they can construct a new kind of cognitive being. Not by examining source code, but by the same oblique, externally mediated, culturally scaffolded process through which every previous cognitive advance has been made.
Which means the doppelganger, if it ever arrives, will be a product of human culture in the fullest sense — not a replacement for the cognitive ranks but their most ambitious expression yet.