Language Models Need Sleep: Architecture and Agent Memory Analysis

Long-lived agents have an awkward memory split. What a model learned before deployment is durable but stale. What it learns inside the current context is fresh but fragile. Retrieval can bring records back into view, yet retrieval alone does not convert experience into reusable parametric skill.

Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories proposes a more radical lifecycle: replace the static train/test boundary with alternating wake and sleep phases. During wake, the learner absorbs new experience through fast-updating memory modules. During sleep, it stops or minimizes external input, transfers recent knowledge into slower parametric memory, and generates synthetic training experiences to improve itself.

This is an architectural metaphor, not evidence that an LLM sleeps or dreams biologically. Its practical contribution is a two-stage update protocol: Knowledge Seeding for memory consolidation, followed by Dreaming for self-improvement.

System lifecycle: wake is for acquisition, sleep is for reorganization

The paper builds on a Continuum Memory System in which MLP blocks update at different frequencies. A fast block is plastic but unstable; a slower block is more persistent but harder to change. Rather than treating memory as a binary short-term/long-term split, stability is relative along a hierarchy.

Figure 1 from the paper contrasts conventional train/test time with a continual learner that alternates wake and sleep. Sleep transfers memories from fast, unstable modules toward slower, more stable components.

The neural oscillation bands in the figure illustrate the biological analogy; they are not operational frequencies for the language model.

A consolidation event occurs before a fast memory block applies an update that could overwrite what it currently stores. The protocol is:

Compute: calculate the prospective base-weight update for the sender block, but do not apply it yet.
Consolidate: add a new low-rank expert to the next, slower memory block. Define the pre-update model as teacher and the expanded model with prospective sender weights as student.
Update: optimize the new expert, apply the sender's base-weight update, reset earlier temporary experts in the fast block, and activate the new expert in the slower block.

Only the newly expanded parameters are trained during this transfer; older student parameters remain frozen. This is intended to isolate new knowledge and reduce interference.

Knowledge Seeding reverses the usual distillation direction

Standard distillation typically moves knowledge from a larger teacher to a smaller student. Knowledge Seeding (KS) moves in the opposite direction: one or more smaller models teach a larger-capacity model. In self-Knowledge Seeding, the teacher and student are two states of the same learner.

The paper combines two objectives:

On-policy generalized distillation: train on a mixture of teacher samples and student-generated sequences, using teacher–student distribution divergence as token-level feedback.
Learning to Imitate (LTI): truncate a teacher-generated sequence, ask the student to continue it, and assign a reward combining semantic agreement and normalized token-edit similarity.

That second component addresses a useful distinction for memory-system design: storing information is not the same as learning a policy that can use it. The student may contain the teacher's knowledge in new parameters yet still sample poorly from it. LTI explicitly trains retrieval-through-generation.

The capacity claim needs careful wording. The appendix says the implementation adds five MLP blocks with dimension 64 and keeps the active parameter count matched to 3B/8B base models. That does not make the total masked or unlockable capacity identical. An implementation can preallocate experts and mask them until a sleep event, avoiding dynamic tensor resizing while still changing which capacity becomes active.

Dreaming is an RL-selected synthetic curriculum

After consolidation, the model enters the Dreaming stage. Given a context and downstream metric, it:

generates multiple synthetic experiences;
routes through an additional random expert to create less obvious combinations;
ranks candidates by gradient-based importance;
keeps top candidates plus random samples for diversity;
applies each dream through LoRA/SFT to an isolated model instance;
rewards a dream if the adapted instance improves the downstream metric;
optimizes dream generation with ReST(^ ext{EM}).

This is not unconstrained recursive self-improvement. The context, task interface, evaluation metric, model family, data-selection rule, and reward are engineered externally. “Without human supervision” means no human labels each generated dream; it does not mean the system is independent of human-designed objectives or datasets.

The most informative ablation: remove Dreaming

The paper's no-context SQuAD experiment measures whether newly incorporated facts remain usable when the passage is no longer present at inference.

Table 3 from the paper reports mean no-context SQuAD accuracy after one passage and after continued pretraining on 200 passages.

The four-level Sleep variant scores 48.9 after one passage and 46.2 after a continued-pretraining run over 200 passages, evaluated on 974 associated questions. SEAL reaches 46.7/43.2, so the reported gains are +2.2 and +3.0 accuracy points.

Removing Dreaming drops the scores to 35.7/36.2. Within this setup, Dreaming is a major contributor. Other ablations are smaller: removing gradient-based selection yields 47.1/45.2, and removing the random expert yields 48.0/44.7.

This table is evidence for the complete pipeline under one experimental protocol, not proof that Dreaming is the sole causal factor. The paper does not report error bars, confidence intervals, statistical tests, or the number of random seeds for these results.

Broader results

The paper evaluates consolidation and the full Sleep process across several tasks:

Mathematical reasoning, average@16: on Qwen3-1.7B, Sleep scores 53.2/40.2/29.3 on AIME-24, AIME-25, and HMMT-25, versus GRPO's 51.0/38.6/26.1. On Qwen3-8B, Sleep scores 79.2/69.0/46.1, versus GRPO's 76.4/68.1/44.9.
Few-shot ARC: Sleep reports 80%, compared with SEAL at 72.5%, TTT at 10%, and ICL at 0%. The protocol is small and filtered: 11 training tasks and eight held-out tasks. The metric is the fraction of dreams that yield a correct answer, not “80% of ARC solved.”
BABILong: the authors report near-perfect performance up to 10 million tokens after benchmark-specific fine-tuning. The appendix explicitly states that all small models, including Sleep, degrade substantially without fine-tuning. This is not evidence of a general-purpose 10M-token context window.
Class-incremental and long-context tasks: Sleep variants outperform the included ICL, EWC, InCA, Hope, DuoAttention, and Cartridges baselines in the reported plots. Those plots do not show uncertainty estimates.

Cost has two valid denominators

At the same number of training steps, the appendix reports that SFT is 4× faster than Sleep. When the comparison instead trains SFT until it matches Sleep's target score, SFT reportedly takes 4.3×, 3.6×, and 4.8× more wall-clock time on AIME-24, AIME-25, and HMMT-25.

Both statements can be true because they answer different questions:

How much does one fixed budget of updates cost?
How much does it cost to reach one target quality?

Production evaluation should report both compute-to-update and compute-to-capability. Quoting only one produces a misleading efficiency claim.

What agent builders should take—and what they should not

The transferable systems idea is not “run fine-tuning every night.” It is to make consolidation a first-class lifecycle phase with explicit boundaries:

define which memory tier is allowed to change;
snapshot the teacher state before destructive updates;
write new knowledge into isolated capacity;
test whether the knowledge is usable, not merely present;
rehearse with synthetic data selected by measurable downstream utility;
prune or reset temporary state only after transfer succeeds.

A production agent would also need controls the paper does not evaluate:

provenance and deletion for consolidated memories;
rollback when synthetic rehearsal corrupts facts or behavior;
safety-drift and model-collapse monitoring;
policies for what deserves consolidation versus forgetting;
storage and serving costs as capacity unlocks over thousands of cycles;
privacy boundaries before user interactions become training data.

The source also has a nomenclature collision: its internal model macro renders as Hope, while Hope is separately listed as a baseline. It is safer to call the proposed method Sleep when interpreting the experiments.

Conclusion

The paper does not demonstrate lifelong autonomous learning. It provides a proof of concept for a more structured update cycle: acquire quickly, consolidate into isolated slower capacity, rehearse through selected synthetic experience, and only then release temporary memory.

For agent architecture, the deeper lesson is that memory is not merely a database or a context window. Durable memory is a transformation pipeline with compute, validation, interference control, and lifecycle policy.

An agent that never pauses to consolidate may accumulate perfect logs and still fail to become wiser from them.

Source

Ali Behrouz, Farnoosh Hashemi, Adel Javanmard, and Vahab Mirrokni, Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories, arXiv:2606.03979v2, 10 July 2026.
The paper is licensed under CC BY 4.0. Figure 1 and Table 3 are reproduced with formatting/cropping changes for web presentation.

Language Models Need Sleep: Knowledge Seeding, Dreaming, and Continual Agent Memory