• Sources: primary, discussion
  • Summary: In a 2026-10-22 post, Cerebras says each WSE-3T wafer has 44 GB of SRAM and each Kimi K3 layer needs about 16 GB at 4 bits. Each wafer holds one dense layer plus its KV cache, or two linear-attention layers, for 59 wafers in total. Only activations move between wafers, over RDMA over Ethernet at about 2 microseconds per hop. Pipeline stages work on different micro-batches at the same time.
  • Why it matters: The design caps concurrent long-context requests by the SRAM left for KV cache on each dense-layer wafer, so read Cerebras throughput claims with that limit in mind.