Google Describes 'Sleep Framework' — LLMs Learn 'While Sleeping'

Google Research published the Sleep framework, which divides LLM operation into an active wake phase and an offline sleep phase. During sleep, knowledge from the context is consolidated into the model's weights via upward distillation, while capacity increases through low-rank experts in MoE.

Compiled by Sergey KostenchukPublished 2026-10-27Updated 2026-10-28

2026-10-27 Research Google

🤖 Google Describes 'Sleep Framework' — LLMs Learn 'While Sleeping'

Google Research published a post on October 27, 2026, accompanying the paper arXiv:2606.03979. In the offline sleep phase, knowledge from the context is consolidated into the weights via upward distillation, while capacity increases through low-rank MoE experts; the dreaming stage fine-tunes the model on synthetic examples. The stated result: reduced catastrophic forgetting and retention of contextual knowledge up to 10M tokens.

🌍 An attempt to solve two systemic problems in production LLMs: knowledge cutoffs and catastrophic forgetting. If the approach works, models will remain relevant longer without retraining.

👤 This is not a ready-made API, but a research paper: details on GKD, LTI, and RL 'dreaming' are in the arXiv version. Nuance: more experts mean more computation; the efficiency gain is a hypothesis.

Source 1: https://research.google/blog/the-sleep-framework-a-new-way-for-llms-to-retain-knowledge-in-continual-learning/ Source 2: https://arxiv.org/abs/2606.03979

Sources