The Beijing ZhiYuan Artificial Intelligence Research Institute, in collaboration with Shanghai Jiao Tong University, Renmin University of China, Peking University, and Beijing University of Posts and Telecommunications, has launched an ultra-long video understanding model named Video-XL. This model is an important demonstration of the core capabilities of multimodal large models and a key step towards General Artificial Intelligence (AGI). Compared to existing multimodal large models, Video-XL shows superior performance and efficiency in processing long videos exceeding 10 minutes.

Recently, the Meta AI team introduced LongVU, a novel spatio-temporal adaptive compression mechanism aimed at enhancing the language understanding capabilities of long videos. Traditional multimodal large language models (MLLMs) face limitations in context length when processing long videos, and LongVU was created to address this challenge. LongVU operates primarily by filtering duplicate frames and employing inter-frame token compression techniques to efficiently utilize context length, allowing it to reduce video data while preserving visual details.

Beijing Zhiyuan Artificial Intelligence Research Institute announces the launch of the native multimodal world model Emu3. This model is based on next-token prediction technology and does not rely on diffusion models or combinatorial methods to achieve understanding and generation across text, image, and video modalities. Emu3 surpasses existing well-known open-source models such as SDXL, LLaVA, and OpenSora in tasks like image generation, video generation, and visual language understanding, showcasing exceptional performance.

Apple Inc. recently launched a significant update for its multimodal AI model MM1, upgrading it to version MM1.5. This upgrade represents more than just a simple change in version number; it is a comprehensive enhancement of capabilities, showcasing the model's stronger performance across various fields. The core upgrade of MM1.5 lies in its innovative data processing methods. The model utilizes a data-centric training approach, carefully selecting and optimizing the training dataset. Specifically, MM1.5 employs high-definition OCR data and synthetic image descriptions, along with optimizations.

Recently, Japan-based startup Rhymes AI unveiled their first artificial intelligence model — Aria. The company claims that Aria is the world's first open-source multimodal mixture of experts (MoE) model. This model not only possesses the capability to handle various input modalities but also claims to be on par with, or even superior to, several well-known commercial models. The design philosophy of Aria aims to provide exceptional understanding and processing capabilities across diverse input forms including text, code, images, and videos.

Recently, Apple's AI research team launched their next-generation family of Multimodal Large Language Models (MLLMs) - MM1.5. This series of models can integrate various data types such as text and images, showcasing new capabilities of AI in understanding complex tasks. Tasks like visual question answering, image generation, and multimodal data interpretation can be better addressed with the help of these models. A major challenge for multimodal models is how to achieve effective interaction between different data types. Previous models often struggled in this aspect.

Meta has recently confirmed that photos and videos taken by users using Ray-Ban Meta smart glasses may be used to train its AI models. This practice has raised concerns as users may not be aware that they are providing Meta with a substantial amount of images for training its AI models, which could include their home interiors, family members, or personal documents. Emil Vazquez, Meta's policy communications manager, stated that in regions where multimodal AI is available, ...

In the digital age, content safety issues are becoming increasingly important. On September 26, OpenAI officially launched a new multimodal content moderation model named 'omni-moderation-latest'. This model is based on the latest GPT-4o technology and can accurately identify harmful text and images. This update will provide developers with powerful tools to build more robust moderation systems. A highlight of the new model is its ability to moderate both text and image inputs, especially when handling non...

Recently, the Allen Institute for AI (Ai2) released Molmo, a brand new open source multimodal AI model family that performs exceptionally well, even surpassing OpenAI's GPT-4o, Anthropic's Claude 3.5, and Google's Gemini 1.5 in several third-party benchmark tests. Molmo not only accepts user-uploaded images for analysis but has also been trained using '1000 times less data than its competitors.'

Recently, an open source multimodal AI model named Molmo has drawn widespread attention in the industry. This AI system, based on Qwen2-72B and leveraging OpenAI's CLIP as the visual processing engine, is challenging the dominance of traditional commercial models with its outstanding performance and innovative features. Molmo's standout characteristic is its efficient performance. Despite its relatively small size, it can compete with competitors that are ten times larger in processing capability. This 'small but exquisite' design philosophy not only enhances the model's

Snapchat recently announced an expanded partnership with Google Cloud, focusing on enhancing the generative AI capabilities of Snapchat's My AI chatbot. Through this collaboration, My AI will be able to leverage the multimodal capabilities of Google's Gemini AI to understand and process a variety of information, including text, audio, images, and video. At Snap's annual partner summit, Snapchat showcased Gemini-powered features.

At a recent press conference, Alibaba International's AI team unveiled its latest development, the multimodal large model Ovis. This innovative AI technology undoubtedly brings new opportunities across various industries. Ovis boasts powerful image understanding and data processing capabilities that are refreshing. Ovis's multimodal ability is exceptionally strong, allowing it to handle various data types such as text and images, showcasing excellent comprehensive strength. Compared to traditional large language models, Ovis not only understands text but also conducts in-depth analysis of non-text information such as images.

OPPO announced that its AI large model AndesGPT-2.0 performed exceptionally in the 'Chinese Large Model Benchmark Evaluation August 2024 Report' released by the authoritative third-party evaluation agency SuperCLUE, winning multiple first-place awards, marking further recognition of OPPO's technological strength in the AI field.

As AI technology continues to advance, understanding user interfaces (UI) has become a key challenge in creating intuitive and useful AI applications. Recently, researchers at Apple introduced UI-JEPA in a new paper, an architecture designed for lightweight, device-side UI understanding that maintains high performance while significantly reducing the computational requirements for UI understanding. The challenge of UI understanding lies in processing cross-modal features, including images and natural language, to capture temporal relationships within UI sequences. Despite the complexity, multimodal large language models...