On the 20th, DeepSeek announced that, based on the previously open-sourced large models, TileLang compilation tools, and inference operator libraries, it has decided to fully open-source its internal inference framework, DeepInfer.
DeepSeek introduces that the capabilities of large models are determined by the models themselves. However, how to provide ultra-large model services at a reasonable cost also depends on an aspect that has not been widely understood by the public for a long time: the reasoning framework.
Training enables the model to achieve results, while inference allows the model to be used by humans. In today's era of massive user requests, the inference framework is responsible for coordinating and scheduling a vast computing power cluster. Its efficiency directly determines the computational resources and energy consumption required for AI services.
DeepSeek has been committed to pushing the boundaries of reasoning performance. "Building on a series of open-source large models, TileLang compilation tools, and inference kernel libraries, we have decided to further open-source our internal inference framework DeepInfer.”
"To do good and to act in accordance with the crowd, to be skillful and achieve success among them." DeepSeek believes it can help developers and enterprise organizations optimize the throughput and latency of large-scale reasoning by sharing internal deployment experience. This enables efficient private cloud and public cloud deployments, reduces computing power costs, and minimizes energy consumption. It also allows intelligent technology to reach a wider audience at lower prices, achieving widespread democratization of large models. "We will continue to support community efforts in optimizing large-scale deployments and welcome you to join our open-source community."
According to the introduction, based on DeepInfer, a large-scale inference service with high throughput and low latency, namely DeepSeek V4.1 Flash, has been implemented. In the future, the inference code for DeepSeek’s new models and subsequent optimizations will also continue to be updated here.
Integrates high-performance operators and deeply optimized inference engines, supporting distributed inference, speculative decoding, SWA Bounded Replay, structured output, etc. Compatible with NVIDIA and Huawei Ascend platforms.
Synchronize open-source production-level Token encoding/decoding components, ensuring full alignment with DeepSeek's official API processing logic, to guarantee the community enjoys an API experience identical to that of the official deployment.
When discussing future prospects, DeepSeek states that with fully open-source models, operator libraries, and inference frameworks, we expect that in the near future, the range and quality of AI services worldwide will experience unprecedented expansion and improvement. These services will become more fair, transparent, and reliable. As a result, technological iteration will accelerate across society, energy waste will be reduced, and limited resources will be utilized more efficiently between infrastructure and user terminals. Intelligent technologies will be accessible to everyone at an affordable cost, bringing with them kindness and endless possibilities.
DeepSeek will work together with everyone to make this future vision a reality with tireless effort.
