The Alibaba Cloud Bairen Model Service Platform recently launched the 'Real-Time Audio and Video Interaction' feature, enabling users to easily build multi-modal AI applications without the need for programming knowledge. This new feature allows users to quickly integrate AI models into Web, iOS, and Android applications and share them with others. Users can create intelligent agent applications through simple steps: first, create a new intelligent agent application, then select and configure the required text, speech, or visual understanding models on the Alibaba Cloud Bairen platform. The platform offers over 200 large models, including

Recently, a research team from the University of Illinois at Urbana-Champaign, Sony AI, and Sony Group introduced a new technology called MMAudio, which aims to achieve high-quality audio synthesis from videos through multi-modal joint training. The core innovation of MMAudio is its ability to generate synchronized audio using video and text inputs, thereby expanding the application scenarios for audio generation. MMAudio is designed to support generating sound effects that align with video content from either video or text inputs.

Amazon has been reported to be developing a multi-modal large language model named 'Olympus', which is expected to be officially announced at the AWS re:Invent conference next week. According to The Information, the internal code name for this algorithm is 'Olympus'. Last November, Reuters reported that Amazon had invested millions of dollars in training a large language model named 'Olympus', with a parameter count reaching 2 trillion.

At last night's press conference, 360 introduced an innovative multi-modal content creation engine called 'Nano Search.' The design philosophy of this engine is simplicity and ease of use, making it suitable for users of all ages, from 5-year-old children to 80-year-old seniors. It enables a one-stop solution for searching, learning, writing, and creating, allowing everyone to become a content creator.

Beijing Aishi Technology Co., Ltd. has announced the global release of its latest version of the video generation large model, PixVerse V3. This version has made significant improvements in its underlying model capabilities, offering multimodal generation abilities and a wealth of AI features, including creative templates, lip synchronization, story continuation, and style transformation.

In today's rapidly advancing field of artificial intelligence, a multi-modal large language model named ORYX is quietly transforming our understanding of AI's ability to perceive the visual world. This AI system, developed collaboratively by researchers from Tsinghua University, Tencent, and Nanyang Technological University, is regarded as the 'Transformers' of visual processing. ORYX, short for Oryx Multi-Modal Large Language Models, is an AI model specifically designed for processing images, videos, and 3D scene time-space understanding.

On September 26, the smart robotics company "Blue Technology" launched the new quadruped robot BabyAlpha A2, stirring a whirlwind in the domestic smart home market with its multi-modal interaction capabilities and approachable price. BabyAlpha A2 is not only an aesthetically pleasing intelligent companion but also a pioneer in home intelligence. As the country's first AI quadruped robot with multi-modal interaction capabilities, it opens the door to smart living for ordinary households with a price of under 10,000 yuan. This robot integrates multiple advanced features, including unique...

On September 21, 2024, Alibaba announced several technological innovations and business developments at the Yunqi Conference in Hangzhou. Among these, the Modao community officially launched the AIGC section, aimed at providing developers with a comprehensive AI creation and development platform. This platform currently offers all functional areas and GPU computing power for free, with the first batch featuring 157 selected multi-modal models, including popular community models and various stylized LoRa models contributed by designers.

MiniMax Xiyu Technology recently announced that since its establishment at the end of 2021, it has built trillion-parameter MoE large text models, speech models, and image models. It has also launched native applications such as Xingye and Hailuo AI, while providing an open platform API service for enterprises and developers.

The Institute of Geochemistry of the Chinese Academy of Sciences, in collaboration with Aliyun, launched the world's first "Lunar Science Multi-modal Professional Large Model" at the 2024 Digital Expo. This model is based on Aliyun's Tongyi series and utilizes RAG retrieval-augmented technology, fine-tuned and trained on Aliyun's Baolian exclusive version, specifically aimed at improving the accuracy of distinguishing the ages and shapes of lunar impact craters, currently exceeding 80%. This innovative model combines visual, multi-modal, and natural language processing technologies, and its optimal application is in identifying impact craters on the moon, which is significant for studying lunar geological evolution.

The latest release from the Alibaba team, mPLUG-Owl3 is a general-purpose multi-modal large model, with its core capability being the understanding of long image sequences. By introducing a hyper attention module, mPLUG-Owl3 can efficiently process visual and language information, achieving in-depth understanding and communication of multi-modal data such as images and videos. This model has made significant breakthroughs in inference efficiency, image processing capabilities, and the application of multi-modal knowledge, particularly in video understanding, where it can 'watch' a 2-hour movie in 4 seconds and accurately answer related questions.

The Sinan OpenCompass team at the Shanghai Artificial Intelligence Laboratory has collaborated with the Modao ModelScope to launch the Compass Multi-Modal Arena, a new section of a large model evaluation platform focusing on multi-modal large models. Users can upload images and input questions to have two anonymous multi-modal large models generate answers, which can then be subjectively evaluated based on the quality of the generated content, allowing users to select the better-performing model. The platform offers an easy-to-use interface and a unique question bank.

ControlMM is an innovative technical framework developed jointly by the Chinese University of Hong Kong and Tencent, aimed at addressing challenges in multi-modal full-body motion generation. This framework supports multi-modal inputs from text, speech, and music to generate full-body motions that match the content. It employs the ControlMM-Attn module to process dynamic and static human topology in parallel, achieving efficient movement knowledge learning. A phased training strategy is adopted, progressing from text to motion pre-training, then to multi-modal control adaptation, ensuring the model's effectiveness under various conditions.

PaddleMIX 2.0 is a multi-modal model development kit launched by Baidu, aimed at simplifying multi-modal application development and supporting scenarios such as autonomous driving, smart healthcare, and search engines. Key highlights include: 1. **Rich Model Library**: Covers images, text, video, and audio, with the addition of LLaVA series models, providing cutting-edge technology support. 2. **Full Process Development Experience**: Equipped with DataCopilot and Auto modules to simplify the training process of multi-modal models. 3. **High Performance Training Capability**

Recently, an international team of researchers from China and Singapore successfully developed the world's first multi-modal artificial intelligence system for diabetes diagnosis and treatment, DeepDR-LLM. The research findings were published in Nature Medicine on July 19, 2024.

In the dazzling galaxy of AI, Leap Star Company has emerged as a shining star with its innovative multimodal models. At the WAIC conference, they showcased three uniquely crafted AI models, highlighting their multimodal capabilities. Step-2: The MoE model with 10 billion parameters is currently available for experience upon application. Step-1.5V, a billion parameter multimodal model, not only demonstrates extraordinary talents in image understanding but also opens up new horizons in video under

Baidu founder Robin Li emphasizes the importance of multi-modal fusion. Baidu has invested in areas such as text-to-video. Baidu is investing in AGI development. The reasoning cost of Baidu's Wenxin large model has been reduced to 1%. Baidu has reached cooperation agreements with Samsung, Honor, and others. Honor will launch its first small foldable phone. Wenxin Yiyan is expected to contribute billions in incremental revenue in 2024. Baidu's investment in text-to-video has made 5G-A a hot topic at the World Mobile Communications Conference.

["Ant Group has launched the 2 billion parameter multi-modal remote sensing foundational model SkySense","SkySense outperforms similar international products across 17 testing scenarios","SkySense can be used for terrain, crop observation, and interpretation","It has been applied in scenarios such as the Alipay Five Fortunes Festival large-scale AI interaction and Ant Group's digital healthcare projects"]