4 releases · 0 this month
v0.32.0 Highlights This release features 814 commits from 363 contributors (121 new)! - Structured decisions and generic parsers: the new POST /v1/systemone endpoint answers choice, noul and score questions over a structured state , is served by default (#60677, #60696), and also runs on Cloudflare/clef-omni , LiquidAI/d1-3B and DiffusionGemma .
v0.31.0 Highlights This release features 717 commits from 307 contributors (96 new)! - DeepSeek-V4.1-Flash performance: FlashMLA mega attention with the V4.1 NVFP4 compressed KV cache is now the SM100 default ; DeepGEMM sparse MQA logits for the indexer and Mega-Gate fusing the gate GEMM with expert selection ; decoder boundaries fuse the TP all-reduce, mHC...
v0.30.0 Highlights This release features 762 commits from 315 contributors (104 new)! New models : DeepSeek-V4.1-Flash ( #56214 , #56228 , #56208 ) with the whole KV stored in MXFP8 through the FlashMLA V4.1 record on SM100 ( #56893 ), DeepGEMM Mega-mHC ( #56962 ), and async Engram prefetch with Engram DP sharding ( #56512 ); DeepSeek-V4-Flash-Vision-Exp (...
v0.29.0 Highlights This release features 594 commits from 277 contributors (91 new)! Model Runner V2 is now the default for all models ( #53183 ), completing the rollout that began with pooling models ( #48290 ).
High-throughput LLM serving.
Releasesby Glyph