Shenzhen Yuanxiang Information Technology Co., Ltd. recently announced the successful release of China’s largest Mixture of Experts (MoE) open source large model — XVERSE-MoE-A36B. The launch of this model marks a significant advancement in China’s AI field, elevating domestic open source technology to an internationally leading level.

The Allen Institute for Artificial Intelligence (AI2) has released an open-source large language model OLMoE, aimed at providing high performance and low-cost solutions. This model employs a Sparse Mixture of Experts (MoE) architecture with 7 billion parameters, but through an intelligent routing mechanism, only 1 billion parameters are used per input token, achieving efficient computation. OLMoE includes both general and instruction-tuned versions, supporting a context window of 4096 tokens. Its training data is sourced from a wide range, including Common Crawl, Dolma CC, and Wikipedia.

At the 2024 Tencent Global Digital Ecosystem Conference, Tencent's Vice President and Head of Cloud Business, Qiu Yuepeng, announced the official launch of the Tencent Hunyuan Turbo large model. This new generation of large models is designed based on the MoE (Mixture of Experts) architecture, achieving a doubling in inference efficiency with a 100% improvement compared to the previous generation, while significantly reducing inference costs by 50%. Additionally, Hunyuan Turbo also demonstrates outstanding performance in decoding efficiency, improving by 2.

MiniMax Xiyu Technology recently announced that since its establishment at the end of 2021, it has built trillion-parameter MoE large text models, speech models, and image models. It has also launched native applications such as Xingye and Hailuo AI, while providing an open platform API service for enterprises and developers.

MoE Technology has announced the open-source release of its independently developed audio understanding large model, MooER, which is the first large-scale open-source speech model trained and inferred on domestically produced full-featured GPUs. MooER was trained on a large-scale audio dataset in just 38 hours on the MoE KuaE Intelligence Computing Platform, demonstrating excellent performance in Chinese and English speech recognition, as well as Chinese-to-English speech translation, achieving an impressive BLEU score of 25.2 in the Covost2 Chinese-to-English test set, nearing industrial-grade effectiveness. MoE Technology plans to further open-source the training code.

MoeThread's AI creation platform 'MoBi MaLiang' announces a complete upgrade, providing users with a more efficient and personalized image generation experience. The upgraded MoBi MaLiang has particularly enhanced its understanding of Chinese language and culture, making creative portrait generation more precise. Users can generate personalized creative portraits quickly in just three simple steps: upload a photo, enter a description, and generate an image.

On January 16, 2024, MiniMax released China's first MoE large language model, abab6. The MoE architecture enables abab6 to handle complex tasks and train more data in a shorter time. abab6 outperforms the previous version abab5.5 in instruction adherence, Chinese comprehension, and English comprehension. abab6 demonstrates excellent capabilities, such as teaching children math problems and helping to build a fictional board game about Shanghai.