HRM-Text: The Hierarchical Reasoning Model Rewriting Pretraining

In April 2026, an independent evaluation confirmed that a 1.15 billion-parameter hierarchical reasoning model scored 60.7% on MMLU and 56.2% on MATH after seeing roughly 40 billion training tokens. That token budget is up to 1,000 times smaller than the 4 to 36 trillion tokens that comparable open models consume. The model is HRM-Text, released by Sapient Intelligence in May 2026, and the numbers are not a rounding error.

For most of the last five years, the recipe for a capable foundation model has been fixed. You gather internet-scale data, you train a transformer to predict the next token, and you spend somewhere between hundreds of thousands and hundreds of millions of dollars doing it. That path put pretraining out of reach for almost every enterprise, and only a handful of labs could afford it.

HRM-Text breaks that assumption in a specific and measurable way. It replaces the single forward-pass transformer with two recurrent stacks that iterate in a continuous latent space before any token is emitted. It also replaces next-token prediction with a task-completion objective trained on structured instruction and response pairs. The result is a model pretrained from scratch in about one day for roughly the price of a used laptop.

This blog explains how the architecture works at the mechanism level, why the cost collapse is real rather than marketing, how the benchmarks should be read with a skeptical eye, and where a compact reasoning core actually belongs inside an enterprise system. It is written for engineers and technical decision makers who need to decide whether this matters to their roadmap this quarter.

What HRM-Text Is and Why the Price Tag Matters

HRM-Text is a 1.15 billion-parameter foundation model built on the Hierarchical Reasoning Model architecture that Sapient Intelligence first introduced in June 2025. The original HRM targeted symbolic reasoning tasks such as Sudoku and the ARC-AGI Challenge, where it beat much larger systems, including DeepSeek R1 and OpenAI o,3 despite having far fewer parameters. HRM-Text is the extension of that architecture intoopen-endedd natural language.

The headline figure that pulled this release into mainstream coverage is the foundation model pretraining cost. Training the 1 billion parameter variant from scratch takes about 46 hours on 16 H100 GPUs across two nodes, at an estimated $1,472 based on a $2 per H100 hour rate. The smaller 0.6 billion parameter variant trains in roughly 50 hours on a single 8-GPU node for about $800. Both figures cover pretraining from random weights, not fine-tuning.

Those numbers matter because they change who is allowed to build. A pretraining run that costs less than a mid-range server invites experimentation from teams that could never justify a seven-figure compute commitment. The architecture uses 130 to 600 times less compute and 150 to 900 times less data than the models it benchmarks against. That is the difference between a research luxury and a line item that a mid-sized company can approve.

At KriraAI, where we build and deliver production AI systems for enterprises, the interesting part is not the record itself. It is what a cheap and honest pretraining loop unlocks for teams that want models shaped around their own data and their own task structure rather than a general-purpose chat assistant.

Inside the Hierarchical Reasoning Model Architecture

The core of HRM-Text is a departure from how a transformer spends its compute. A standard transformer processes a sequence in a single forward pass, layer by layer, and its depth of computation is fixed by its layer count. The hierarchical reasoning model instead runs two coupled stacks that loop over the same embeddings, adding computational depth through recurrence rather than through more parameters.

Dual Timescale Recurrence

The architecture is built from two modules that operate at different speeds. The high-level module, called H, handles slow and abstract planning. The low-level module, called L, handles fast and detailed refinement. In a single forward pass, the model performs 2 H updates and 6 L updates, for a total of 8 stack iterations, with the L stack updating more frequently than the H stack.

This multi-timescale structure is the mechanism behind latent space reasoning. The model thinks in its internal hidden state across these iterations before it commits to any output. Depth of reasoning becomes a function of recurrence steps rather than physical layers. That is why a compact model can behave like a much deeper one on hard problems while staying small enough to deploy on a phone.

The separation between the two modules is not cosmetic. Sapient found that a single shared recurrent stack, the kind used in some tiny recursive models, becomes unstable when scaled to a billion parameters for language. As the team put it, language needs both fast local refinement and slow semantic stability, and forcing one module to do both breaks training. The two timescales exist because the math demands them.

Additive State Injection and Shared Blocks

Information moves between the two stacks through simple state addition at every recurrent step. In inference mode, the loop reads cleanly. The low-level state is updated as L of the sum of the low-level state and the high-level state, and the high-level state is then updated as H of the sum of the high-level state and the new low-level state.

Both stacks share the same underlying transformer block design. Each block uses gated attention, rotary position embeddings, a SwiGLU feed-forward layer, and pre-RMSNorm. In other words, HRM-Text does not throw away the transformer. It wraps proven transformer blocks inside a recurrent control structure that reuses parameters across iterations, which is how it reaches effectively unbounded compute depth at a bounded parameter count.

The PrefixLM Attention Mask

HRM-Text is pretrained with a PrefixLM objective rather than a pure causal one. Prompt tokens attend to each other bidirectionally, while response tokens attend causally to everything before them. This matters in practice because it treats the instruction as a fully visible condition block and the answer as the thing to generate.

Anyone serving the model needs to respect this mask. If you drop the model into a standard causal inference framework without preserving the bidirectional prefix, output quality degrades. Serving stacks such as vLLM can handle it, but the attention masking has to be configured correctly. This is one of several places where the operational discipline required is real, even though the model is small.

Task Completion Training: Learning to Finish Work, Not Predict Words

The second pillar of HRM-Text is its training objective, and it is as important as the architecture. Conventional foundation models learn by predicting the next token across raw text. Every word carries equal weight in that setup, so filler phrases, function words, and critical reasoning steps are all treated the same. HRM-Text abandons that in favor of task completion.

The 40 Billion Token Diet

HRM-Text was trained from scratch on roughly 40 billion tokens of structured data, which is where the up to 1,000 times data reduction comes from. The corpus is not scraped internet prose. It is composed of instruction and response pairs spanning general instructions, math, symbolic logic, textbook exercises, and rewritten knowledge. The model learns to complete tasks, not to autocomplete sentences.

This is the mechanism that makes the small data budget defensible rather than suspicious. When the training signal is a task and its solution, each example teaches structure and rules rather than surface phrasing. The model learns to derive patterns from similar task structures quickly. That efficiency is why 40 billion curated tokens can stand in for trillions of raw ones on reasoning-heavy work.

Stripping the Thinking Tokens

Sapient made one deliberate and counterintuitive choice during data preparation. They explicitly removed visible chains of thought or thinking tokens from the training data. The goal was to force the model to perform its reasoning inside the hierarchical recurrent loop rather than by copying step-by-step traces on the page.

This is the practical face of latent reasoning models, where reasoning happens internally instead of relying on long visible chains of thought.  Instead of spending large output token budgets on long visible reasoning chains, HRM-Text does the heavy lifting internally and emits a more direct answer. For latency-sensitive and cost-sensitive deployments, reasoning that happens in the hidden state rather than in the generated text is a genuine advantage, because you pay for fewer output tokens per solved problem.

HRM-Text vs Transformer LLMs: What Actually Differs

The HRM-Text vs transformer comparison is easy to overstate, so it helps to be precise about what changes and what stays the same. The building blocks are shared. The control flow, the training objective, and the scaling philosophy are not.

The clearest differences are worth listing directly:

• A standard transformer fixes computational depth by layer count, while HRM-Text extends depth dynamically through recurrence steps at a fixed parameter count.

• A standard LLM scales its capability mainly by adding parameters and data, while HRM-Text reuses parameters through its dual timescale loop to add reasoning depth without growing the model.

• Conventional pretraining optimizes next-token prediction over raw text, while HRM-Text optimizes task completion over structured instruction and response pairs.

• Mainstream reasoning models externalize thought as long visible chains, while HRM-Text performs reasoning in a continuous latent space and emits shorter outputs.

None of this makes the transformer obsolete. HRM-Text vs transformer is not a replacement story; it is a specialization story. The recurrent core is strong at bounded, rule-driven reasoning and weak at broad factual recall, precisely because it was never asked to memorize the internet. That trade is the whole point, and understanding it is how you decide where the model belongs.

Benchmark Performance and How to Read It Honestly

The benchmark scores are striking, and they deserve a careful rather than a breathless reading. In an independent verification conducted in April 2026, HRM-Text achieved 56.2% on MATH, 82.2% on DROP, 81.9% on ARC-Challenge, and 60.7% on MMLU. These results reflect the base model only, with no post-training, fine-tuning, or reinforcement learning applied.

Placed against models trained on orders of magnitude more data and compute, those figures are competitive on reasoning-heavy tasks. The right way to interpret them is that a curated 40 billion token diet can produce a strong reasoning engine, even though it cannot produce a broad knowledge oracle. The MATH and ARC-Challenge scores speak to structured reasoning. The MMLU score, while respectable, sits where you would expect for a model that was deliberately not trained to memorize wide world knowledge.

The team is candid about the boundary. Coding is a clear weak spot because HRM-Text was not trained on code datasets, and early base scores on coding benchmarks landed in the low single digits. Notably, third-party experiments that fine-tuned on roughly 1 billion tokens of code data pushed coding scores up to around 40 to 50, which suggests the architecture adapts well once given the right domain data. The honest reading is that benchmark headlines describe reasoning potential, and each production capability still has to be earned through targeted adaptation.

Where the Hierarchical Reasoning Model Fits in Enterprise Systems

For enterprise teams, the real question is placement. A 1.15 billion-parameter reasoning core is not a drop-in ChatGPT replacement, and treating it as one guarantees disappointment. It is a component, and it fits a specific set of slots very well.

Offline and Edge Deployment

HRM-Text runs locally with a 0.6 GiB footprint at int4 quantization, which means it fits comfortably on a modern smartphone or a modest edge device. If you're evaluating efficient on-device AI deployment, understanding modern LLM quantization techniques is equally important for reducing memory usage while preserving model accuracy. This is where the architecture earns its keep. For any workflow where sensitive data must not leave the premises, a compact model that reasons on device rather than in the cloud is a structural advantage rather than a convenience.

This edge profile lines up directly with the systems KriraAI builds for regulated and privacy-sensitive clients, where processing data locally is often a hard requirement rather than a preference. A reasoning core small enough to run on an agent device, a kiosk, or an on-premises server changes what is architecturally possible for offline-first products.

The Reasoning Core Plus Memory Pattern

The most important design insight is the decoupling of reasoning from knowledge. HRM-Text proves that a model does not need to memorize the internet to become a capable reasoning engine. For enterprise use, this is a feature because it points to a clean architecture: a compact recurrent reasoning core paired with an external knowledge layer.

That knowledge layer is where retrieval-augmented generation and learned memory modules return. You pair HRM-Text with a vector store that supplies facts, and let the recurrent core reason over retrieved evidence and business rules. This reasoning core plus memory pattern suits enterprise question answering, compliance logic, and structured extraction, and it is a pattern KriraAI already applies when matching model choice to a client workload.

Adaptive Computation at Inference

Because reasoning depth is controlled by recurrence steps, there is a natural lever for cost control at inference. Easy prompts can halt early after fewer iterations, while hard prompts can reserve the full recurrent budget. This adaptive reasoning approach is particularly valuable in AI for Financial Services, where fraud detection, risk assessment, and regulatory workflows require balancing reasoning quality with inference cost.

Adaptive computation time is not fully productized in this initial release, but the architecture makes it a reachable optimization rather than a research fantasy. For a serving fleet, spending full recurrence only on the queries that need it is a meaningful efficiency win.

The Honest Limitations You Need to Price In

Sapient is refreshingly direct that the first HRM-Text release is a proof of concept, closer to an early GPT release than a finished product. Any serious adoption plan has to budget for real engineering, and pretending otherwise leads to failed pilots. The following constraints are the ones that matter most.

• The published checkpoint is a pre-alignment base model. It has not been instruction-tuned, dialogue-tuned, long-context-adapted, or RLHF-aligned, so assistant-style behavior requires your own supervised fine-tuning or reinforcement learning.

• Broad factual recall is intentionally weak. The model reasons well but knows relatively little, which makes an external retrieval or memory layer mandatory for knowledge-heavy applications.

• Coding is poor out of the box, since the base model saw no code data, and closing that gap requires domain-specific fine-tuning.

• The training corpus is predominantly English, so multilingual and code-switching workloads, which matter a great deal in markets like India, will need additional data and adaptation.

• Training targets Hopper-class GPUs such as the H100 because the attention path depends on FlashAttention 3, so the cheap pretraining figure assumes access to that specific hardware class.

None of these are disqualifying, but together they define the work. The operational effort centers on templates, mode selection, attention masking, and alignment, as the team itself describes it. A compact model lowers the cost of the model, not the cost of the system around it.

What Enterprise Teams Should Do Now

The pragmatic response to HRM-Text is neither to ignore it nor to rip out your existing stack. It is to run a scoped evaluation on the workloads where a compact reasoning core plays to its strengths. The path to a credible internal assessment is fairly concrete.

1. Identify one bounded, reasoning-heavy workflow with clear rules, such as multi-step financial reasoning, compliance checking, or structured extraction followed by a decision.

2. Pull the open-weight checkpoint from Hugging Face under its Apache 2.0 license and reproduce the reference benchmarks to confirm the model behaves as documented in your environment.

3. Fine-tune on a small, well-curated set of your own instruction and response pairs for that single workflow, using condition tags to control direct versus reasoned output styles. Organizations evaluating compact foundation models often rely on Machine Learning Development Services to fine-tune architectures, optimize training pipelines, and deploy production-ready models tailored to their business requirements.

4. Wire the tuned core to a retrieval layer so factual knowledge comes from your systems of record rather than from the model's weights.

5. Serve it with a PrefixLM-aware inference stack and measure latency, accuracy, and unit cost against your current approach on the same task.

Because the entire pretraining loop is open-sourced, including data extraction, sequence packing, and distributed training, teams with unusual data or strict privacy needs can even pretrain a variant from scratch rather than only fine-tune. The ability to train a foundation model from scratch on a controlled corpus, for roughly a thousand dollars, is genuinely new leverage for organizations that were previously locked out of pretraining entirely. The strategic move is to learn where a small reasoning core beats a large general model on your own workloads before your competitors do.

Conclusion

Three things are worth carrying away from this. First, on how it works, HRM-Text earns its efficiency through a hierarchical reasoning model architecture that reasons in latent space across dual-timescale recurrent loops and trains on task completion rather than next-token prediction. Second, where it matters, the model shines as a compact, offline-capable reasoning core paired with an external knowledge layer, not as a general-purpose chat replacement. Third, on what to do, the right move is a scoped evaluation on one bounded, reasoning-heavy workflow before broad commitment.

At KriraAI, we build and deliver production AI systems for enterprises, and we track the research frontier closely so that we adopt new techniques when they are genuinely ready to produce measurable value rather than when they are merely new. An architecture that lets a company train a foundation model from scratch for the price of a laptop, orfine-tunee a private reasoning core that never sends data to the cloud, is exactly the kind of development we assess against real workloads rather than hype cycles. HRM-Text is early, its limitations are real, and its strategic implications for cost, privacy, and control are equally real.

If your team is weighing whether a compact reasoning core belongs in your stack, we would welcome a conversation about what this architecture could mean for your specific workloads. The organizations that learn where small, specialized models beat large general ones, on their own data and tasks, will hold a durable advantage as this shift plays out.