Daniel Kahneman distinguished two thinking systems: System 1 (fast, automatic, intuitive, error-prone) and System 2 (slow, deliberate, analytical, reliable). This distinction maps precisely onto current AI systems. And it explains why these systems are simultaneously so impressive and so unreliable.
System 1: The Most Brilliant Intuition Machine Ever Built
An LLM generates text through next-token prediction. Each token is an implicit decision based on statistical patterns extracted from billions of texts. No conscious rule application, no planning, no reasoning — just pattern recognition in high-dimensional spaces.
This is functionally identical to what Kahneman describes as “intuition”: fast, automatic judgments that are usually correct but produce systematic errors.
And the list of capabilities is impressive:
- Pattern recognition: LLMs detect connections in texts that humans miss. They find the needle in the haystack, as long as the haystack is made of text.
- Context completion: Give an LLM three sentences from a specialized domain, and it completes the fourth in an appropriate register, with fitting vocabulary, at the right depth.
- Analogy formation: The ability to recognize structural similarities across different domains (“Validation Gates are to data what an immune system is to an organism”) is among the strongest capabilities of current models.
- Social intuition: LLMs can read tone, mood, and social dynamics in texts and respond accordingly. Not because they feel, but because they have internalized the statistical patterns of human communication.
- Creative association: Combining ideas that a human would not have combined, because they lie in different fields between which no single person sees all the connections.
All of this is System 1. Fast, automatic, usually correct. And this is precisely where the problem lies.
What System 1 Cannot Do
Kahneman wrote an entire book about how systematically System 1 errs. The same errors appear in LLMs — not coincidentally, but structurally:
Causal reasoning. System 1 recognizes correlations, not causation. “After” becomes “because of.” LLMs reproduce this pattern reliably. They can explain why A and B are connected, but the explanation is a plausible story, not a logical proof. And plausible stories are the most dangerous thing there is, because they feel right.
Metacognition. System 1 does not know what it does not know. It delivers an answer and has no way to assess its quality. LLMs are identical: they produce an answer with the same confidence whether it is correct or wrong. Confidence is not encoded in the output, because it is never computed during the process.
Epistemic sensitivity. Where does this come from? How certain am I? Is my claim based on one data point or a thousand? System 1 does not ask these questions. Neither do LLMs. A fact from a single questionable source is presented with the same matter-of-factness as a mathematical identity.
Anticipation. System 1 reacts. It does not anticipate. It cannot say: “Before you ask this question — you will probably also want to know X.” That would require a model of the questioner and a model of its own knowledge gaps. Both are missing.
The Reasoning Illusion
Chain-of-thought prompting and reasoning models simulate slow thinking through enforced intermediate steps. The model “shows its work.” But: more tokens are not automatically deeper thinking.
The debate is instructive. Is a reasoning model that “thinks” for 10,000 tokens really closer to System 2? Or is it a more sophisticated System 1 that traverses more patterns without ever performing an external reality check?
Kahneman’s System 2 has a decisive property: it checks against reality. It says “Wait, is that actually true?” and searches for counterexamples. A reasoning model does not do this. It generates a longer chain of plausible-sounding intermediate steps, but the plausibility comes from the same statistical patterns as the original answer.
A deterministic piece of code that validates a claim against an external database is closer to Kahneman’s System 2 than a model that extends its own chain. That sounds counterintuitive, but it is the logical consequence of Kahneman’s definition.
The Gap
The gap between System 1 and genuine System 2 is not “more parameters” or “better training.” It is an architectural gap. What is missing:
-
A persistent model of its own state. The system must know what it knows and what it does not. Not as a probability distribution over tokens, but as an explicit representation.
-
External validation. The system must be able to check its outputs against something external — facts, sources, contradictions. Not through more thinking, but through looking things up.
-
Temporal awareness. When was a piece of information last confirmed? Is it still current? System 1 has no sense of time. System 2 needs one.
This is precisely the space in which the Selbstvektor (self-vector) operates: a compact self-model that gives the system the ability to weight its own information processing. And the Validation Gates implement the external verification authority that System 1 lacks.
The thesis is not that LLMs are bad. They are the best implementation of System 1 ever built. The thesis is that System 1 alone is not enough. And that the solution does not lie in larger models, but in an architecture that adds System 2 as an independent layer.
References
- Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux. ISBN 978-0-374-27563-1.
- Tversky, A. & Kahneman, D. (1974). Judgment under Uncertainty: Heuristics and Biases. Science, 185(4157), 1124–1131. DOI: 10.1126/science.185.4157.1124
- Stanovich, K. E. & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate. Behavioral and Brain Sciences, 23, 645–665. DOI: 10.1017/S0140525X00003435
- Evans, J. St. B. T. & Stanovich, K. E. (2013). Dual-Process Theories of Higher Cognition: Advancing the Debate. Perspectives on Psychological Science, 8(3), 223–241. DOI: 10.1177/1745691612460685
- Wei, J. et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS 2022. arXiv: 2201.11903
- Li, Z. et al. (2025). From System 1 to System 2: A Survey of Reasoning Large Language Models. arXiv: 2502.17419
- Griot, M. et al. (2025). Large Language Models lack essential metacognition for reliable medical reasoning. Nature Communications, 16, 642. DOI: 10.1038/s41467-024-55628-6
- Geng, J. et al. (2024). A Survey of Confidence Estimation and Calibration in Large Language Models. NAACL 2024, 6577–6595. DOI: 10.18653/v1/2024.naacl-long.366