explainx / blog
Mandy Lu (Stanford PhD, Google AI) ignites discussion on X by stating 'we still have no satisfying theory for why AI works'—exposing the gap between transformers' empirical success and our theoretical understanding of scaling laws, emergent abilities, and mechanistic interpretability.
Jul 16, 2026
Researcher @anthrupad ran a simple probe on Claude Fable/Mythos — ask for your top 3 favorite video games, 10 times — and the results went viral (~24.9K views). Two games were fixed attractors, the third slot varied, and the model's self-analysis reads like a field guide to what frontier LLMs find meaningful.
Jul 14, 2026
Anthropic's July 14 pledge funds eight Canadian research partnerships with Claude API credits, adds Amii/Mila/Vector to its startup program, and publishes Canada's first Economic Index country brief — 2.6% global share, second-highest per-capita adoption after the US.
Jul 14, 2026
The father of reinforcement learning left John Carmack's Keen to launch Oak Lab — algorithms that learn from runtime experience, not curated datasets. Sutton's OaK architecture targets continual world models and 20W planning. explainx.ai explains the AGI neolab wave and the Bitter Lesson tension.
On June 3, 2026, Mandy Lu—a Google AI researcher with a Stanford PhD in computational mathematics—posted a simple statement on X (formerly Twitter) that ignited a viral debate:
"we still have no satisfying theory for why AI works"
The post struck a nerve. With over 80 reactions and hundreds of replies, it exposed an uncomfortable truth in the AI research community:
Despite transformers powering ChatGPT, Codex, and every major AI breakthrough since 2017, no one fully understands why they work so well.
We have scaling laws that predict performance. We have mechanistic interpretability that maps features. We have emergent abilities that appear unpredictably.
But we don't have a unified theory explaining why massive models trained on internet-scale data can reason, code, and create.
It's like building a rocket to the moon using thermodynamics—but without understanding atoms.
| Topic | Key Facts |
|---|---|
| The Problem | No satisfying theory explains why transformers trained on massive datasets perform reasoning, coding, and creative tasks so effectively. |
| Scaling Laws | Predict performance gains from compute, data, and parameters—but not the underlying reasons why scaling works. |
| Emergent Abilities | Capabilities (reasoning, in-context learning) appear in large models unpredictably; some debate if emergence is real or measurement artifact. |
| Mechanistic Interpretability | Reverse-engineers neural networks to map features and pathways; named MIT's 2026 Breakthrough Technology. |
| Current State | AI works empirically (thermodynamics) but lacks fundamental theory (atomic physics). Practical results prioritized over theory. |
| Debate Context | Sparked by Mandy Lu (Google AI, Stanford PhD) on X; reflects broader tension in AI research community. |
Mandy Lu is an AI researcher at Google working on health and climate applications. She holds a PhD in Computational and Mathematical Engineering from Stanford University, admitted Autumn 2025.
Her academic background includes:
Her research projects focus on:
Her interdisciplinary background in computational mathematics, neuroscience, and AI makes her uniquely positioned to identify gaps between empirical results and theoretical understanding.
On June 3, 2026, Lu posted:
"we still have no satisfying theory for why AI works"
The simplicity and directness of the statement resonated across the AI research community, tech workers, and skeptics alike.
Replies ranged from:
The discussion exposed a fundamental tension in AI research: we can build increasingly powerful systems, but we can't fully explain them.
Before exploring what we don't understand, let's establish what we do know.
Since the "Attention Is All You Need" paper (Vaswani et al., 2017), transformers have dominated AI:
The architecture works—undeniably and reproducibly.
Research from OpenAI (2020) and DeepMind's Chinchilla (2022) established that model performance follows power-law relationships with:
You can predict the performance of a 100B parameter model trained on 2T tokens before you train it.
As models scale, new capabilities appear that weren't present in smaller models:
These abilities emerge at certain scale thresholds—but we can't predict exactly when or why.
Researchers can now:
MIT Technology Review named mechanistic interpretability one of its 10 Breakthrough Technologies for 2026.
Despite these advances, fundamental questions remain unanswered.
The self-attention mechanism is the core of transformers. It allows the model to:
We know how it works mathematically. We can implement it. We can optimize it.
But why does this particular mechanism enable reasoning, creativity, and generalization?
Scaling laws are descriptive, not explanatory.
They tell us:
They don't tell us:
As one researcher put it: "Scaling laws are like the ideal gas law. They predict behavior, but they're not a fundamental theory."
Emergent abilities are the most mysterious phenomenon in AI.
At some scale threshold, models suddenly:
Why?
Some researchers argue emergence is real—a phase transition in the model's internal representations.
Others argue it's a measurement artifact: we're using crude metrics that fail to capture gradual improvements in smaller models, making progress look sudden when it's actually continuous.
A 2026 paper proposed that LLMs are non-ergodic systems where capabilities emerge through discrete transitions guided by constraint interactions—but this is still a hypothesis, not a proven theory.
Models are trained on:
Somehow, this leads to models that can:
Why does next-token prediction on internet text lead to general reasoning abilities?
As one X commenter put it:
"Tech VCs thought AI was sentient because they had never read books and hence the LLM was beyond anything they had ever seen."
The implication: LLMs might just be extremely good pattern matchers, not true reasoners. But if that's the case—why do they generalize so well?
Multiple replies to Lu's post invoked a historical analogy:
"We're using AI like we used thermodynamics before we understood atomic theory."
In the 1800s, engineers built steam engines using thermodynamics. They could:
But they didn't understand why thermodynamics worked until the kinetic theory of gases and statistical mechanics explained heat and pressure in terms of molecular motion.
Similarly, AI researchers today can:
But we lack the fundamental theory that explains why transformers work at a mechanistic, first-principles level.
One camp of researchers argues we're making progress toward a theory through mechanistic interpretability.
Mechanistic interpretability reverse-engineers neural networks to understand how AI actually thinks. It aims to uncover how a model computes outputs by analyzing:
Anthropic has led this field with several major advances:
Anthropic announced a "microscope" that identified features corresponding to recognizable concepts:
Anthropic traced whole sequences of features and the path a model takes from prompt to response, showing:
Researchers demonstrated selective control of model behavior by:
Anthropic has stated its goal: "Reliably detect most AI model problems by 2027 using interpretability tools."
Recent research suggests knowledge is encoded as geometry in high-dimensional space.
Models represent concepts as vectors, and relationships between concepts correspond to geometric relationships (distances, angles, subspaces).
This explains:
But why does gradient descent on next-token prediction lead to these semantically meaningful geometric structures?
That's still an open question.
Another reply thread focused on scaling laws as a partial theory.
The OpenAI Scaling Laws paper (2020) and DeepMind's Chinchilla paper (2022) established:
Loss decreases as a power law with:
Optimal allocation of compute requires balancing model size and data:
Emergent abilities correlate with scale:
Scaling laws are phenomenological: they describe what happens, not why.
They don't explain:
A 2026 unified framework connected scaling laws to in-context learning emergence, showing that ICL performance follows power-law relationships with model depth, width, context length, and training data—but the exponents are determined by task structure.
This is progress toward theory, but still descriptive, not first-principles.
Some replies pushed back on the premise, arguing AI doesn't work as well as claimed.
One X commenter wrote:
"I would be way more bullish on AI if it actually worked and was actually replacing real humans at scale. Nothing is changing and we're being lied to. The tools don't work! They are expensive! And high maintenance. Dot com bubble 2.0."
This reflects growing AI skepticism as enterprises face:
Others countered:
The debate reflects a gap between hype and reality—but also genuine progress amid unrealistic expectations.
Some replies drew parallels to neuroscience, where we:
One commenter noted:
"We know how AI works. We don't fully know why it works as well as it does. That's an important distinction."
This mirrors neuroscience:
Similarly:
A pragmatic camp argued: Who cares about theory if it works?
One reply stated:
"The real-world deployments showed augmentation, not replacement. Humans plus AI is the winning formula."
This reflects the engineering mindset: prioritize building useful systems over understanding fundamental mechanisms.
Historically, this has worked:
But in each case, theory eventually caught up and enabled:
Understanding why AI works isn't just academic—it has practical implications.
If we don't understand why models produce certain outputs, we can't:
Anthropic's interpretability research aims to solve this by 2027, but we're not there yet.
Understanding why scaling works could help us:
State Space Models and Mixture of Experts architectures are attempts to move beyond transformers, but they're still empirical experiments, not theory-driven designs.
Understanding why pre-training on internet data leads to general reasoning could help us:
A theoretical understanding could:
So where are we now?
2026 breakthroughs include:
But none of these constitute a complete, first-principles theory.
Three scenarios:
AI research develops a unified theory (like statistical mechanics for thermodynamics) that explains:
This enables theory-driven AI design and predictable capabilities.
Likelihood: Medium. Mechanistic interpretability is making progress, but we're not close to a unified theory yet.
AI systems keep improving through:
Theory lags behind, but practical results drive adoption.
Likelihood: High. This is the current trajectory.
Without theoretical understanding, we:
AI progress slows dramatically as empirical scaling plateaus.
Likelihood: Low-Medium. Some evidence of diminishing returns emerging, but not a hard wall yet.
If you're building with AI, here's what Lu's observation means:
Without a theory, you can't fully predict when models will:
Design systems with human oversight and fallbacks.
Since theory can't predict behavior, test extensively:
Mechanistic interpretability tools are improving. Follow:
If a theoretical breakthrough happens, it could:
Mandy Lu's statement—"we still have no satisfying theory for why AI works"—is both alarming and accurate.
We've built systems that:
But we can't fully explain why they work.
This isn't just an academic curiosity. Without theory, we:
We're flying blind on the most powerful technology of the 21st century.
The good news: progress is happening. Mechanistic interpretability, scaling law research, and geometric theories are advancing. Anthropic aims to "reliably detect most AI model problems by 2027."
The bad news: we're not there yet. And in the meantime, billions of dollars and critical decisions depend on systems we don't fully understand.
For developers, the lesson is clear: build with humility. AI is powerful, but unpredictable. Test rigorously. Keep humans in the loop. Stay updated on interpretability research.
And watch for the breakthrough—when it comes, it could change everything.
AI theory and interpretability research evolve rapidly. This analysis reflects the state of knowledge as of June 2026. For the latest research, follow Anthropic Research, OpenAI Research, and leading ML conferences.
