• Sources: primary
  • Summary: In the sleep phase the model unlocks pre-allocated MoE blocks, distills short-term memory into them with on-policy distillation and RL, and resets temporary parameters. A dreaming stage generates and filters synthetic rehearsal data. Google reports near-zero forgetting on continual translation of Manchu and Kalamang and near-perfect BABILong scaling to 10M tokens.
  • Why it matters: It is a concrete design for updating a deployed model with new knowledge without full retraining or catastrophic forgetting.