• Sources: primary, discussion
  • Summary: The model shipped 2026-10-03 with full weights on Hugging Face under Apache 2.0, which makes it commercially usable rather than research-only. The architecture is 78.1B total parameters with 3.46B active per token, 50 layers that are all MoE with 384 experts and 6 active plus one shared, a 128,000-token vocabulary, sliding 512-token attention with full attention every fifth layer, context up to 1M, and four reasoning effort levels. Training ran on 768 B200 GPUs in Germany and Finland with 21 days of pre-training at 20T tokens, German at about 21.3% of the pre-training mix, translated text held to 6%, and 38 unplanned interruptions handled by automatic restart from a checkpoint at most 250 steps back.
  • Why it matters: An Apache 2.0 release at this scale with a disclosed training recipe is usable in commercial products, though every benchmark figure is the vendor's own and unreproduced.