explainx / blog
A technical guide to Hierarchical Reasoning Models (HRM) and Tiny Recursive Models (TRM): architecture, training tricks, ARC-AGI results, and what recursive inference changes for reasoning systems.
Jul 14, 2026
Anthropic's July 14 pledge funds eight Canadian research partnerships with Claude API credits, adds Amii/Mila/Vector to its startup program, and publishes Canada's first Economic Index country brief — 2.6% global share, second-highest per-capita adoption after the US.
Jul 14, 2026
The father of reinforcement learning left John Carmack's Keen to launch Oak Lab — algorithms that learn from runtime experience, not curated datasets. Sutton's OaK architecture targets continual world models and 20W planning. explainx.ai explains the AGI neolab wave and the Bitter Lesson tension.
Jul 12, 2026
James Evans analyzed 41.3M papers — individual AI adopters soar, collective curiosity shrinks. explainx.ai maps Goodhart dynamics, paper mills, and whether frontier model wins change the picture.
Most AI scaling conversations still default to one strategy: bigger models, more data, longer context. HRM and TRM added a different axis in 2025: more recursive computation at inference time without proportionally increasing parameter count.
This post summarizes the key ideas from recent HRM/TRM research and the Decoded discussion transcript you shared, then maps those ideas to practical model-design choices.
| Question | Short answer |
|---|---|
| What changed? | HRM/TRM showed that small models can gain strong reasoning behavior via recursive latent refinement loops. |
| Why was it interesting? | Reported ARC-style results were strong relative to model size and training data. |
| Core mechanism | Reuse the same weights repeatedly over internal states (z, z_low, or equivalent) at inference and training. |
| HRM key idea | Two-timescale recursion (high-level + low-level modules). |
| TRM key idea | Simplify to one tiny shared network and keep recursive refinement + deep supervision. |
| Big takeaway | Inference-time recursion is a meaningful compute axis, not just parameter scaling. |
Primary sources:
A transformer forward pass is highly parallel and efficient for training, but many reasoning tasks are effectively multi-step algorithms. If the task is hard to compress into one pass, performance can bottleneck even when the model is large.
In the transcript, this is framed as a gap between:
That distinction matters. Token-space traces are useful, but they are constrained by discrete outputs and supervision artifacts. Latent recursion can keep iterative computation inside a continuous state space.
Hierarchical Reasoning Model (HRM) proposes two interacting recurrent modules:
At a high level, training repeatedly:
Reported results in the paper include strong performance on reasoning-heavy tasks (including ARC-style settings) with a relatively small parameter budget and limited training samples.
Reference: Wang et al., 2025
Tiny Recursive Model (TRM) keeps the core recursive refinement intuition but simplifies architecture and training design.
From the paper’s framing:
The paper reports that this simplified setup outperformed HRM on key ARC-AGI metrics while using fewer parameters.
Reference: Jolicoeur-Martineau, 2025
A useful way to reason about the difference:
| Approach | Iteration medium | Typical failure mode |
|---|---|---|
| Chain-of-thought | Tokens (external text) | Verbose traces, brittle decomposition, inherited token errors |
| Tool-use loops | Tokens + external API calls | Bounded by tool availability and prior knowledge |
| HRM/TRM recursion | Continuous latent state | Training stability and optimization details become central |
This does not make chain-of-thought obsolete. It reframes it as one recursion interface, not the only one.
ARC-style problems emphasize abstraction and stepwise transformation. They are often hard to solve via a single direct mapping from input to output.
Recursive latent refinement is naturally aligned with these tasks because it allows:
That is the core reason these papers attracted attention: not just scoreboards, but a different compute strategy.
If you are building reasoning systems, these papers suggest a practical design checklist:
Separate model capacity from compute depth Capacity (parameters) and iterative depth (recursion steps) should be tuned independently.
Treat recursion loops as first-class hyperparameters Refinement steps, supervision depth, and state-reset behavior can matter as much as width/depth.
Benchmark for algorithmic generalization, not only text fluency Include tasks where single-pass pattern matching fails.
Expect hybrid architectures General-purpose pretrained models plus compact recursive reasoning heads/modules is a plausible near-term direction.
Important caveats:
The most realistic interpretation is not “small recursive models replace frontier LLMs.”
It is: recursive inference is a complementary scaling law. The field can continue scaling pretrained world models while adding stronger latent recursive computation where algorithmic reasoning is the bottleneck.
That matches where many labs are heading across agent systems and reasoning stacks: combine broad priors with targeted iterative computation.
This article is based on:
Paper results and benchmark standings can change with revised evaluations, replications, and new benchmark versions. Verify against the latest arXiv revisions and ARC Prize updates.
