Skip to content- Sources: primary, discussion
- Summary: DeepInfer is Apache-2.0 licensed and supports DeepSeek-V4.1-Flash with attention DP, MoE expert parallelism, draft-token decoding, batch-invariant inference, deterministic sampling and structured outputs. The release covers offline inference only. It leaves out the KV cache system, prefill and decode disaggregation, and load balancing needed for a full service. The README says it was originally built on vLLM.
- Why it matters: Teams serving DeepSeek models on Blackwell-class or Ascend hardware get a reference engine with deterministic modes. They still need their own serving layer.