• Sources: primary, discussion
  • Summary: Ubicloud measured 92 percent schema accuracy on Moonshot's K2VV ToolCall benchmark when serving Kimi K2.6 on vLLM 0.28, and 91 percent after vLLM made constrained decoding opt-in. Enabling XGrammar structural-tag constrained decoding via vLLM's --tool-strict-level flag gave 100 percent for Kimi and DeepSeek models. vLLM keeps it off by default because of model divergence, grammar translation bugs, and interaction bugs with speculative decoding.
  • Why it matters: Teams self-hosting open-weight models for agents should check the strict tool-call setting before blaming the model for failed tool calls.