Paper reports low-bit quantization inflates reasoning-model output length

Category: Paper
Status: developing
Sources: arXiv 2606.25519
Summary: A preprint posted 2026-06-25 argues that quantizing reasoning models to low bit widths inflates the number of reasoning tokens they emit, so the extra generation length partly offsets the memory and bandwidth savings quantization is meant to deliver. The authors present this token inflation as a hidden cost not captured by accuracy-at-fixed-budget comparisons. Results are the authors' own and not independently reproduced.
Why it matters: If it holds, teams choosing quantization for reasoning-model serving need to budget for longer outputs, not just smaller weights, when estimating latency and cost.
Follow-up: Watch for independent reproduction across model families and quantization schemes.