Large Language Models (LLMs) like GPT and Llama have sparked a revolution in the field of artificial intelligence, but efficiently training these massive models while aligning them with human values remains a challenge. Reinforcement Learning with Human Feedback (RLHF) has become an important training method for LLMs in recent years, but traditional RLHF frameworks have limitations in flexibility, efficiency, and scalability. To address these issues, the ByteDance Doubao Large Model Team has open-sourced an RLHF framework called HybridFlow.

ByteDance quietly enters the new battlefield of AI communities, drawing industry attention with its internally incubated AI platform, Lumi. Positioned as a community for sharing AI models, the platform is still in the whitelist testing phase, but its complete ecological layout is beginning to take shape. As an important part of ByteDance's strategy in the AI field, Lumi's core positioning is to create a comprehensive platform that integrates model sharing, workflow building, and model training. In the context of AI technology rapidly penetrating various industries, the emergence of this platform may provide developers and researchers with greater opportunities.

Recently, ByteDance quietly launched an AI model sharing community called 'Lumi'. The platform integrates various functions including model sharing, workflow building, and LoRA training, although it is currently still in the internal testing phase. Product entrance: https://top.aibase.com/tool/lumi. The core highlight of Lumi is its model sharing and management features, allowing users to easily upload their developed AI models.

On October 31, ByteDance quietly launched an AI model sharing community platform called 'Lumi'. The platform is reported to offer features such as model upload and sharing, workflow building, and LoRA training. It is currently in the internal testing phase and is only accessible to whitelisted users.

Recently, ByteDance officially addressed rumors regarding an 'intern disrupting large model training'. The company confirmed that an intern did maliciously interfere with the model training tasks of the commercialization technology team's research project. However, this action did not affect the company's official commercial projects and online business, nor did it involve other large model businesses of ByteDance. Moreover, ByteDance pointed out that claims circulating online about 'involving over 8,000 nodes and causing losses of over ten million dollars' are seriously exaggerated. According to public reports, the intern was part of ByteDance's commercialization technology team.

At today's 2024 Volcano Engine AI Innovation Tour, in addition to the video generation model, ByteDance also launched the Doubao Music Model and Doubao Simultaneous Interpretation Model, announcing significant upgrades to the Doubao General Model Pro, Text-to-Image Model, Speech Synthesis Model, and other specialized models. The introduction of the Doubao Music Model signifies Volcano Engine's deep commitment to the field of music creation. Supported by powerful algorithms, this model enables high-quality music creation freely. For lyrics generation, it can quickly generate emotional lyrics based on just a few simple input words.

At the 2024 Volcano Engine AI Innovation Roadshow, Volcano Engine President Tan Dai unveiled the latest Doubao · Video Generation Model. This model incorporates several advanced technologies, including the efficient DIT Fusion Computing Unit, which enables deep compression encoding of video and text; it also employs a novel diffusion model training approach to ensure consistency in generating multi-camera videos; in addition, the model integrates a deeply optimized Transformer architecture, significantly enhancing the generalization ability of video generation.

Today, ByteDance's Volcano Engine announced that the Doubao Large Model will release a video generation model on September 24, bringing upgrades to more capabilities in the model family. It is understood that the Doubao Large Model was officially launched at the Volcano Engine Original Power Conference on May 15, 2024.

Recently, foreign media reported that ByteDance is accelerating its efforts to self-develop and manufacture AI chips to reduce dependence on expensive chips from NVIDIA. The report claims that ByteDance plans to collaborate with TSMC and expects to mass-produce two self-developed AI chips by 2026, having already reserved the production of hundreds of thousands of chips. More notably, the report pointed out that ByteDance aims for its self-developed chip clusters to achieve four times the computing performance of a single H100 chip at a similar cost. In response, ByteDance quickly addressed this report.

Recently, the foreign media outlet The Information reported that ByteDance is accelerating the development of its own AI chips and plans to collaborate with TSMC, aiming for mass production by 2026. The report also stated that this move by ByteDance is intended to reduce dependence on NVIDIA chips and lower the costs of developing and operating AI models. Source note: The image is generated by AI and provided by the image licensing service Midjourney, according to reports from the Sci-tech Innovation Board Daily regarding today's media coverage of ByteDance's plans to collaborate with TSMC on AI chips.

Recently, Doubao, the educational application from ByteDance's AI smart assistant brand, completed a major upgrade. This app aims to be a smart partner for learners and a valuable assistant for family education, providing support for the new term's studies. 'Doubao Aixue' has introduced multiple auxiliary learning features, including answering questions, homework correction, knowledge Q&A, essay guidance, and mental calculation practice, to meet the diverse needs of family education.

ByteDance's open-source FLUX Dev's Hyper SD Lora has revolutionized AI drawing speed with its efficient image generation capabilities, shortening the traditional, lengthy process to just 8 or 16 steps. This move not only provides a fast track for creative realization but also establishes a demonstration space on Hugging Face, allowing users to directly experience the drawing effects of the 8-step Lora. Although there is some degradation in image quality, it has minimal impact for most users. Comparison examples show that the degradation is not significant when not enlarged.

ByteDance is secretly preparing to establish a Large Model Research Institute to strengthen its position in AI large models. The institute has attracted top talents from the industry, including Qin Yujia and Huang Wenhao, who will manage technology projects and report directly to founder Zhang Yiming. This move indicates ByteDance's long-term plans and proactive strategies in AI model R&D, aiming to leverage external expertise for technological a....

The third edition report of 'Top 100 Consumer Generative AI Applications' released by Andreessen Horowitz reveals popular AI applications and their usage trends. The report shows that content generation tools, particularly in image, video, and music, dominate web applications, with 52% of web products focusing on this area. The music generation tool Suno has notably improved its ranking. In the mobile application sector, image and video editing tools are favored by users, and ChatGPT remains...

At the Volcano Engine AI Innovation Roadshow in Shanghai on August 21, 2024, Volcano Engine showcased a comprehensive upgrade of the Doubao Large Model. This includes the Doubao Text-to-Image Model, which has improved text and image matching capabilities for long texts, the Doubao Speech Recognition Model, which reduced error rates by up to 40% across multiple public test sets, and upgrades to the Doubao Speech Synthesis Model, enhancing streaming speech synthesis abilities for real-time responses and accurate punctuation. Volcano Engine also released a real-time interactive solution for Conversational AI, integrating the Doubao Large Model with real-time audio and video technology, providing end-to-end capabilities.

The Seed-ASR engine launched by ByteDance achieves high-precision recognition of Mandarin, 13 Chinese dialects, and 7 foreign languages through massive training data, significantly enhancing the convenience of cross-language communication. Its key advantage lies in its excellent contextual awareness, accurately recognizing proper nouns, place names, and keywords by incorporating historical information, especially performing exceptionally well in specific scenarios, thereby improving recognition accuracy. Whether in daily conversations, complex meetings, or interactions among multiple people in noisy environments, Seed-ASR can transcribe accurately. It can also recognize various professional terms.

ByteDance and a research team from Shanghai University of Science and Technology have developed an innovative technology called 'HeadGAP' that can quickly generate highly realistic and animatable 3D virtual head models using just three photos taken from different angles, enabling facial expression synchronization. During the research process, the team first extracted 3D head prior information from a large multi-view dynamic dataset to customize the virtual avatar for the target individual. By utilizing a Gaussian point cloud-based self-decoding network and partial dynamic modeling, in conjunction with inversion and fine-tuning strategies, they efficiently achieved high-quality and diverse...

Today, ByteDance announced a new feature for the Doubao model that supports real-time voice calls. It is reported that the conversational AI real-time interaction solution provided by Volcano Engine combines the Volcano Ark large model service platform and Doubao's speech recognition and synthesis models, streamlining the conversion process between speech and text. This solution achieves efficient voice data collection, processing, and transmission, offering outstanding intelligent dialogue and natural language processing capabilities.

The LSLM full-duplex language model developed in collaboration between Shanghai Jiao Tong University and ByteDance revolutionizes traditional AI assistant interaction by enabling real-time conversation where users can listen while speaking. The core advantage of LSLM lies in its ability to simultaneously process speech input and generate output, significantly enhancing the natural flow of dialogue. By integrating streaming self-supervised learning with a token-based decoder, this model supports real-time speech interaction and performs exceptionally well even in noisy environments. LSLM's mid-term fusion strategy strikes the optimal balance between speech generation and real-time interaction, demonstrating impressive noise resistance.