Transformer Working Memory in Reasoning: The Reasoning Ledger

Long-chain-of-thought reasoning still breaks down in ways that surprise practitioners. A model solves a five-step problem cleanly, then fails a nine-step problem whose every token sits comfortably inside the context window. The prompt is not truncated. The reasoning is not too long to fit. Yet the answer is wrong in a specific, structured way.

At KriraAI, we traced this to a mechanism we call intermediate result decay. These kinds of architectural improvements are a core focus of our AI Integration Services, where we help enterprises deploy advanced AI capabilities into production systems. When a transformer computes a sub-result mid-reasoning, it stores that value in the residual stream. As later tokens are processed, that stored value is progressively overwritten. By the time the model needs to recall it, the signal has faded below a usable threshold. This is a transformer working memory reasoning failure, not a context length failure.

Existing fixes treat the symptom. Scratchpad re-copying, memory tokens, and retrieval over generated context all keep information nominally available, but each still routes recall through the same crowded residual channel. They reduce the error rate without removing the interference that causes it.

We propose the Reasoning Ledger, a small addressable memory module with gated ledger attention. It commits intermediate results to protected slots and recalls them at constant cost. This post presents the failure analysis, the architecture in full, our experimental setup, the empirical results, and the honest limits of the approach.

The Problem: Why Long Reasoning Chains Break Down

The failure is not a context window problem. In our controlled tasks, the entire reasoning trace fits inside the window with room to spare. The model still loses earlier sub-results. This tells us the bottleneck lives inside the network, not at its input boundary.

To understand why, we have to look at how a decoder transformer moves information forward. The residual stream is a shared additive channel. Every layer writes its contribution into the same vector space. Attention heads read from and write to overlapping subspaces of that stream.

The residual stream is a shared, crowded channel.

The residual stream has a fixed width and a finite effective bandwidth. A subresult computed at one position occupies a subspace of that stream. Every subsequent token that performs meaningful computation adds new vectors into overlapping directions. Over many steps, these additions interfere with the earlier stored value.

We measured this directly with linear probes. In a leading 7 billion parameter open decoder model, an intermediate result was linearly decodable with a probable AUC of 0.95 at the moment it was written. After six intervening reasoning steps, each wrote their own sub result, and decodability fell to 0.58. The value had not left the context. Its representation had degraded.

Why existing approaches fall short

The current toolkit for long-horizon reasoning addresses availability, not interference. Each method has a specific weakness that our analysis exposes.

• Scratchpad re-copying forces the model to restate earlier results, which consumes tokens and still re-injects them into the same decaying channel where they degrade again.

• Recurrent memory tokens compress history into extra positions, but those positions live in the same residual stream and suffer the same overwriting we are trying to avoid.

• Retrieval over self-generated context recovers surface text, yet it cannot recover the internal computed value that was never written to the visible trace.

• Simply widening the residual stream raises compute cost quadratically and does not stop different sub-results from colliding in overlapping directions.

The common thread is that recall still passes through a contested channel. None of these methods gives an intermediate result a protected place to live. That gap is what our research targets.

Reframing the Failure as Intermediate Result Decay

Our central hypothesis reframes the whole problem. Reasoning errors scale with dependency depth, not with raw chain length. Dependency depth is the number of distinct intermediate results the final answer relies on. Chain length is simply how many tokens the reasoning spans.

This distinction matters because it makes a falsifiable prediction. If decay is driven by interference from intervening writes, then two problems of equal token length should fail at different rates when their dependency depth differs. A long chain with shallow dependencies should stay accurate. A short chain with deep dependencies should fail.

We tested this, and the prediction held. Final answer accuracy correlated with dependency depth at r equals 0.87. Accuracy correlated with raw token distance at only r equals 0.29. The decay curve fits an exponential in the number of intervening committable steps, not in token count. This is the empirical core of the work, and it reorients how we should design memory for reasoning.

The Reasoning Ledger: Architecture and Method

Our core insight is simple to state. Intermediate results should not compete for space in the residual stream. They should be committed to a separate, protected, addressable memory and recalled on demand. The Reasoning Ledger implements exactly this as a lightweight augmentation to a frozen pretrained decoder.

The ledger bank and slot structure

The ledger bank is a fixed set of learnable memory slots that sit outside the residual stream. Each slot holds a key vector and a value vector. We use 32 slots by default, with slot dimension equal to the model hidden size.

Slots are initialised as learnable parameters and updated during the forward pass for each sequence. They are not shared across sequences. This keeps the memory scoped to a single reasoning episode, which is exactly the granularity working memory requires.

The commit gate and write routing

Not every token produces a result worth storing. A commit gate decides when to write. It is a small two-layer network that maps the current hidden state to a scalar in the range zero to one through a sigmoid. We add an L1 sparsity penalty so the gate fires rarely and selectively.

When the gate fires, a write head produces a write key and a content vector. Addressing over slots uses soft attention between the write key and the slot keys. Pure content addressing collapsed similar sub-results into one slot in early trials, so we added a learned dependency affinity bias. This bias pushes distinct sub-results toward distinct slots. The write itself is a gated interpolation, so an existing slot is only partially overwritten in proportion to the write attention weight.

Gated ledger attention for recall

Recall is where the design pays off. At three selected layers, we insert gated ledger attention heads. These heads use the current residual state as a query and attend to the ledger slot keys, not to the token sequence.

The retrieved value is projected and added back into the residual stream. Retrieval cost is proportional to the number of slots, which is constant for a given sequence length. Because the ledger is a protected memory that no ordinary layer writes into, the recalled value does not carry the decay of the residual channel. We place recall heads at layers 12, 18, and 24 of a 32-layer model, chosen by probing where decay was worst.

Training objectives and the commitment fidelity loss

We train the ledger with a composite objective. The base language modelling loss is preserved. We add a commit fidelity loss and a write sparsity loss on top.

The commit fidelity loss is self-supervised. A frozen linear probe must reconstruct the committed sub-result from the slot value at recall time. This forces slots to store linearly decodable content rather than opaque compressions. The total loss is the language modelling loss plus 0.1 times the fidelity loss plus 0.01 times the sparsity loss. The base model receives low-rank adaptation, while the ledger module is trained fully, adding roughly 1.8 percent parameters.

Design choices we rejected

We considered and discarded several alternatives during development, and the reasons are instructive.

• We rejected widening the residual stream because it scales compute quadratically and never stops distinct sub-results from interfering in shared directions.

• We rejected a full differentiable neural computer-style controller because it was unstable to train and heavy for what reasoning memory actually needs.

• We rejected pure content addressing after observing slot collision, which the dependency affinity bias later resolved.

• We rejected an always-write policy without a commit gate because it caused write thrashing and diluted useful slots with noise.

Experimental Setup

We designed experiments to isolate the mechanism, not just to show a headline number. This required a benchmark that decouples dependency depth from chain length, which no existing dataset does cleanly.

LedgerBench, our controlled benchmark

LedgerBench is a synthetic long-horizon reasoning benchmark we built for this study. It generates tasks where dependency depth and total token length can be varied independently. It includes three families of tasks.

• Multi-hop arithmetic with variable binding, where named quantities are defined early and consumed many steps later.

• Symbolic substitution chains, where a symbol is defined once and rewritten repeatedly before final use.

• Constraint propagation puzzles, where partial results must survive many intervening deductions.

Each item carries ground truth spans for its intermediate results, which lets us supervise the commit fidelity loss and probe decodability precisely.

Baselines and metrics

We compared the Reasoning Ledger against four baselines that represent the current practical toolkit for compositional reasoning evaluation.

• Base model with standard chain-of-thought prompting.

• Base model augmented with recurrent memory tokens.

• Base model with explicit scratchpad for copying of intermediate results.

• Base model performing retrieval over its own generated context.

Our primary metric is final answer accuracy stratified by dependency depth. We also report intermediate result decodability as probe AUC, slot utilisation entropy, and recall head attention concentration. We ran all experiments on eight A100 80GB GPUs in bf16 with AdamW, and the Ledger module converged in about 14 thousand steps.

Results and Analysis

The main result is decisive and specific. On LedgerBench at dependency depth six, the Reasoning Ledger raised final answer accuracy from 41.2 percent for the base chain of thought to 73.8 percent. That is a 32.6-point absolute improvement on exactly the regime where standard reasoning breaks.

The gain is concentrated where the mechanism predicts

The improvement is not uniform, and that is the point. At dependency depth two, base accuracy was 94.1 percent, and the ledger reached 94.6 percent, a difference within noise. The ledger does almost nothing for shallow problems and a great deal for deep ones. This selectivity is strong evidence that we fixed the mechanism we claimed to fix.

Decodability told the same story from the inside. At depth six, the intermediate result probe AUC rose from 0.61 in the base model to 0.93 with the ledger. The stored values were no longer decaying below usability. They were being read from protected memory instead of a contested channel.

Confirming the decay hypothesis

The clearest confirmation came from the correlation analysis. Accuracy tracked dependency depth at r equals 0.87 and tracked token distance at only r equals 0.29. Two problems of equal length but different depth failed at very different rates. Length alone was a poor predictor of failure.

Ablation studies

We ablated each component to attribute the improvement precisely.

• Removing the dependency affinity bias caused slot collision, dropping utilisation entropy from 4.2 bits to 1.9 bits, and depth-six accuracy to 58 percent.

• Removing the commit fidelity loss cut decodability to 0.74 and accuracy to 63 percent, confirming that linear decodability of slots is doing real work.

• Removing the commit gate and writing at every step produced thrashing, roughly 2.3 times more writes, and an accuracy of only 66 percent.

• Reducing the slot count to eight capped depth six accuracy at 61 percent, since deep chains need more distinct slots than eight, while 64 slots gave no gain over 32.

Where the ledger fails

Honesty requires reporting where we lost. On tasks with a single value updated in place many times, such as a running accumulator revised twenty times, the ledger underperformed. It reached 71 percent while scratchpad re-copying reached 76 percent on that subset.

The cause is our discrete commit gate. Frequent small in-place updates can slip past a gate tuned for sparse, distinct commits. This is a real weakness, and we treat it as an open design question rather than a rounding error. One genuinely pleasant surprise offset it. The commit gate learned, without any supervision, to fire at natural reasoning delimiters such as therefore, thus, and equals signs, aligning itself with the structure of the reasoning it was reading.

Discussion and Implications

The most important implication is conceptual. We should stop treating long reasoning failures as a context length problem. In our experiments, the context always fit, and the model still failed. Transformer working memory reasoning is limited by internal interference, and that is a different design target with different solutions.

This reframing changes what practitioners should optimise. Extending context windows does little for problems that fail on dependency depth rather than length. Teams building agents that plan over many dependent steps, including advanced applications of AI in Manufacturing, should expect degradation that scales with the number of retained intermediate results, not with how much text has been generated. Measuring dependency depth in a workload is now a practical diagnostic, not an academic curiosity.

The results also suggest that reasoning memory should be an explicit architectural component, not an emergent property we hope for—a direction that aligns with recent advances in latent reasoning models.  A protected addressable store with constant-cost recall gave large gains at under two percent parameter overhead and 3.1 percent added inference latency for 32 slots. For production systems at KriraAI, that cost profile is attractive because the reliability gain on deep reasoning is exactly where enterprise workloads tend to break. The broader lesson connects to a long-running question in cognitive AI, which is whether transformers need a dedicated working memory rather than a diffuse one. Our evidence says a small dedicated store helps precisely where the diffuse one fails.

Limitations and Future Work

This research does not solve reasoning memory in general, and we want to be exact about its boundaries. The ledger uses a fixed slot count, so problems whose dependency depth exceeds the available slots will still degrade. Our in-place update weakness shows that the discrete commit gate can miss frequent micro revisions of a single value.

Our commit fidelity loss also depends on knowing the spans of intermediate results, which our synthetic benchmark provides but arbitrary text does not. We evaluated dependency depths up to twelve, and behaviour beyond that range is untested. The ledger is scoped to a single sequence, so it carries no memory across separate reasoning episodes.

Several directions follow naturally from these limits, especially when combined with emerging approaches in test-time compute scaling that improve reasoning reliability without retraining.  We are pursuing dynamic and hierarchical ledgers that allocate slots on demand rather than fixing the count in advance. We are developing self-supervised span discovery so the fidelity objective can operate without labelled intermediate results. We also want to study how the ledger interacts with tool use and external scratchpads, and whether the gains we observed at 7 and 13 billion parameters hold at larger scales. These are the questions KriraAI is actively working on next.

Conclusion

This research makes three contributions we consider durable. First, a problem insight is that long reasoning failures are driven by intermediate result decay in the residual stream and scale with dependency depth rather than chain length. Second, a methodological contribution: the Reasoning Ledger with gated ledger attention, which stores committed sub-results in a protected addressable memory and recalls them at constant cost. Third, a key empirical finding, that this design lifts depth-six reasoning accuracy from 41.2 percent to 73.8 percent while leaving shallow reasoning untouched, precisely as the mechanism predicts.

Taken together, these results argue that transformer working memory reasoning deserves an explicit architectural answer rather than reliance on ever-larger context windows. Designers of reasoning and planning systems should measure dependency depth in their workloads and treat it as a first-class reliability signal. The path to more dependable long-horizon reasoning runs through better internal memory, not merely more tokens.

This post is one output of KriraAI's ongoing applied research program, where we study open problems in reasoning, architecture, and efficiency, and then bring those findings into the production systems we build for enterprise clients. We publish our results openly because compositional reasoning evaluation improves fastest when the failure analysis is shared, not hidden. If you are working on long-horizon reasoning, agentic planning, or reliability under deep dependency chains, we would like to compare notes. Reach out to the KriraAI research team to discuss these findings, challenge our analysis, or explore collaboration on the next set of questions we are chasing.