• Sources: primary, discussion
  • Summary: The post distils Aleph Alpha's 189-page technical report, the model card, and the launch post into six design choices: 50 layers of 384 experts plus one shared expert with 6 routed per token, giving 78.1B total and 3.46B active parameters, a 128,000-token UniBPE vocabulary, sliding-window attention on 40 of 50 layers with only those layers carrying rotary position embeddings, which is what lets context stretch past the 262,144 tokens it was trained on to a validated 1,048,576, German-language reasoning training, a per-request reasoning_effort selector, and Merlin-Arthur abstention training. The author ran his own tokenizer comparison over the German Basic Law and measured 15 percent fewer tokens than the GPT-5 tokenizer, a larger reduction than the 11.2 percent Aleph Alpha claims rather than a reproduction of it, and he also lists the weak rows: last of twelve on a closed-book test, 39.8 on multi-turn Berkeley Function Calling against 54.0 for Qwen3.5 35B-A3B, and 27.7 on Terminal-Bench 2.1. Benchmark comparisons otherwise come from Aleph Alpha's own evaluation, and the author discloses a friendship with a member of the team.
  • Why it matters: The tokenizer measurement is the one number in the post a reader can reproduce independently, and it lands above the vendor's figure on a corpus the author chose himself.