Shanghai AI Lab Releases Large Model Training Toolbox XTuner, Significantly Reducing Training Costs

ByteDance Open-Sources COMET: A Technology Boosting Large Model Training Efficiency by 1.7x

ByteDance's Doubao large model team recently announced a breakthrough in addressing key bottlenecks in Mixture-of-Experts (MoE) architecture, open-sourcing a significant optimization technology called COMET. This technology dramatically improves large model training efficiency, achieving a remarkable 1.7x speedup and a 40% reduction in training costs. Image Note: Image generated by AI, image licensing provider Midjourney. COMET has been deployed in ByteDance's multi-thousand-GPU cluster training, resulting in millions of GPU hours saved.

Vivo Restructures, Establishes New AI Department and Shifts Large Model Training to On-Device

DeepSeek Open Source Release Day Four: Parallel Strategy Upgrade with DualPipe and EPLB Technologies Revolutionizes Large Model Training

Tencent Hunyuan Releases New Theory on Floating Point Quantization Training, Revealing the Limits of Large Model Training

In today's rapid development of Large Language Models (LLM), the training and inference costs of models have increasingly become a focus of research and application. Recently, the Tencent Hunyuan team released an important study that delves into the 'Scaling Laws' of low-bit floating point quantization training, which refers to the principles governing the scale of floating point quantization training. The core of this research lies in exploring how to significantly reduce computational and storage costs without sacrificing performance by lowering the precision of the model.

"Buddhist" Optimizer C-AdamW: One Line of Code to Boost Large Model Training Speed by 1.47 Times!

ByteDance Unveils Open Source Secret Weapon HybridFlow, Boosting Large Model Training Speed by 20 Times and Slashing Costs!

Large Language Models (LLMs) like GPT and Llama have sparked a revolution in the field of artificial intelligence, but efficiently training these massive models while aligning them with human values remains a challenge. Reinforcement Learning with Human Feedback (RLHF) has become an important training method for LLMs in recent years, but traditional RLHF frameworks have limitations in flexibility, efficiency, and scalability. To address these issues, the ByteDance Doubao Large Model Team has open-sourced an RLHF framework called HybridFlow.

ByteDance Responds to 'Intern Disrupting Large Model Training': No Impact on Official Commercial Projects

Recently, ByteDance officially addressed rumors regarding an 'intern disrupting large model training'. The company confirmed that an intern did maliciously interfere with the model training tasks of the commercialization technology team's research project. However, this action did not affect the company's official commercial projects and online business, nor did it involve other large model businesses of ByteDance. Moreover, ByteDance pointed out that claims circulating online about 'involving over 8,000 nodes and causing losses of over ten million dollars' are seriously exaggerated. According to public reports, the intern was part of ByteDance's commercialization technology team.

Volcano Engine Launches Video Preprocessing Solution for Large Model Training, Adopted by PixelDance

At a recent video cloud technology conference, Volcano Engine unveiled a significant innovation: a video preprocessing solution for large model training. This technology has been successfully applied to the Doubao video generation model, marking a significant advancement in AI video generation technology. Volcano Engine's President, Tan Dai, emphasized that AIGC and multimodal technology are profoundly changing user experience. Leveraging experience from Douyin, Volcano Engine is actively exploring the integration of AI large models with video technology to provide comprehensive solutions for enterprises. Wang Yue, Head of Video Architecture at Douyin Group, pointed out that large model training faces numerous challenges.

PaddlePaddle Framework 3.0 Introduces Unified Automatic Parallelism, Simplifying Large Model Training Development

The release of PaddlePaddle Framework 3.0 features a core upgrade to unified automatic parallelism technology, aimed at simplifying the distributed training process for large models and improving development efficiency. The new version supports mixed parallelism techniques from four-dimensional to five-dimensional, employing various parallel methods such as data parallelism, tensor model parallelism, pipeline parallelism, and grouped parameter slicing parallelism, effectively enhancing the training efficiency of large models. The automatic parallelism technology uses tensor slicing syntax to automatically infer distributed slicing states and add communication operators, reducing the difficulty of development. The principles of automatic parallelism include distributed tensor representation and slicing inference.

Tencent Cloud Unveils Star Pulse Network 2.0, Accelerating Large Model Training by 20%

Tencent Cloud recently launched an upgraded version of the Xing Mai Network 2.0, aiming to enhance the efficiency of large-scale model training. In the previous version, the synchronization communication time for the computation results of large models accounted for over 50% of the total, resulting in low efficiency. The new version of Xing Mai Network 2.0 has been upgraded in several aspects:1. Supports networking of up to 100,000 cards in a single cluster, doubling the scale, with a 60% improv