Skip to content- Sources: primary, discussion
- Summary: The llama-modchip repository says it shrinks DeepSeek-V4-Flash to 56.56 GiB with a custom block-64 format and MoE expert pruning, reaching about 44 tokens per second prefill and 9 to 10.7 tokens per second decode on a 16-core Zen 5 CPU, with GGUF files that unpatched llama.cpp cannot load. The Hacker News thread drew 241 points and 72 comments.
- Why it matters: Claims of frontier MoE models on commodity CPUs depend on quantization below 4 bits, so quality needs checking on your own tasks before adoption.
- Comments: Commenters cite a TUM paper finding the best size-to-quality ratio near 4 bits, while others counter that quantization-aware training and large MoE models degrade less. One commenter traces the README's 76.87 GiB baseline to an IQ1_S file mislabeled as IQ2_XXS and calls the result likely heavily degraded. Others say llama.cpp's backlog of about 1,700 pull requests pushes such work into forks.