Professor Zhai Jidong's team at Tsinghua University's Institute for High-Performance Computing, in collaboration with Tsinghua-affiliated startup Qingcheng Extreme Intelligence, recently announced the open-source large model inference engine, "Chitu." This innovative technology marks a significant breakthrough for China in the AI field, particularly in inference engine development.
"Chitu" excels in its ability to natively run FP8 precision models on GPUs outside of Nvidia's Hopper architecture and various domestic chips. This means developers, regardless of whether they rely on imported or domestically produced chips, can leverage this inference engine for efficient AI model inference. This advancement not only promotes the widespread application of domestically produced AI chips but also provides strong technical support for the development of China's AI ecosystem.
Image Source Note: Image generated by AI, image licensing provided by Midjourney
Tang Xiongchao, CEO of Qingcheng Extreme Intelligence, stated at the launch that China currently lacks production-ready open-source products in this core area of inference engines. The open-sourcing of "Chitu" aims to address this pain point and contribute to the development of China's AI ecosystem. By opening the source code, developers and researchers can freely use, modify, and optimize this inference engine, accelerating the iteration and innovation of AI technology in China.
Amidst the increasingly fierce global competition in AI technology, this initiative by the Tsinghua team will provide new opportunities for Chinese companies in the AI sector. With the "Chitu" inference engine, companies can develop AI products that meet market demands more quickly, enhancing their overall competitiveness. Furthermore, the open-source approach will attract more developers to participate in AI research and development, collectively driving technological progress.
The release of "Chitu" is not only a technological innovation but also a step towards improving China's domestic AI industry chain. In the future, we expect more companies and developers to utilize this tool to make waves in the AI world and usher in a new era of AI in China.