Alibaba's Tongyi Qianwen Open Source Qwen2.5 Series Models: Qwen2-VL-72B Comparable to GPT-4

AIbase基地

Published inAI News · 3 min read · Sep 19, 2024

1.7k

The Tongyi Qianwen team has announced the open-source release of the latest member of the Qwen family, the Qwen2.5 series language models, three months after the release of Qwen2. This marks one of the largest open-source releases in history, including the general-purpose language model Qwen2.5, as well as specialized models for programming and mathematics, Qwen2.5-Coder and Qwen2.5-Math.

The Qwen2.5 series models have been pre-trained on a massive dataset containing up to 18T tokens. Compared to Qwen2, the new models show significant improvements in knowledge acquisition, programming capabilities, and mathematical abilities. They support long text processing, capable of generating up to 8K tokens, and maintain support for more than 29 languages.

WeChat Screenshot_20240919080229.png

WeChat Screenshot_20240919080238.png

The open-source release of the Qwen2.5 series models not only adopts the Apache2.0 license but also offers various scale versions to meet different application needs. Additionally, the Tongyi Qianwen team has also open-sourced the Qwen2-VL-72B model, which matches the performance of GPT-4.

The new models have achieved significant improvements in instruction execution, long text generation, structured data understanding, and generating structured outputs. Especially in the fields of programming and mathematics, the Qwen2.5-Coder and Qwen2.5-Math models, trained on specialized datasets, demonstrate stronger capabilities in their respective domains.

Experience the Qwen2.5 Series Models:

Qwen2.5 Collection: https://modelscope.cn/studios/qwen/Qwen2.5

ZhiYuan and Tencent Launch Long Text Understanding Benchmark Model LongBench v2

On December 19, 2024, at a press conference, ZhiYuan Research Institute and Tencent announced the launch of LongBench v2, a benchmark designed to evaluate the deep understanding and reasoning capabilities of large language models (LLMs) in real-world long text multi-task scenarios. The platform aims to advance long text models in understanding and reasoning, addressing the current challenges faced by large language models in practical applications.

AI Unicorn Cohere Deeply Collaborates with Palantir: Enterprise-level AI Applications Emerge

As one of the most watched AI startups outside of OpenAI and Anthropic, Cohere was valued at $5.5 billion in July. One of the company's co-founders is an author of the paper 'Attention Is All You Need,' which is considered critical to sparking the revolution of large language models (LLMs). Headquartered in Toronto and San Francisco, Cohere focuses on providing AI solutions for enterprise clients, rather than pursuing a similar path as other companies.

Nexa AI Launches OmniAudio-2.6B: A Fast Audio Language Model for Edge Deployment

Nexa AI recently unveiled its new OmniAudio-2.6B audio language model, designed to meet the efficient deployment demands of edge devices. Unlike traditional architectures that separate automatic speech recognition (ASR) and language models, OmniAudio-2.6B integrates Gemma-2-2b, Whisper Turbo, and a custom projector into a unified framework. This design eliminates the inefficiencies and delays associated with linking various components in traditional systems, making it especially suitable for resource-constrained computing.

Meta Launches 'Large Concept Models' (LCMs)! Breaking Through LLM Limitations and Leading a New Direction in AI Language Understanding

Large Language Models (LLMs) have made significant progress in the field of Natural Language Processing (NLP), shining in applications such as text generation, summarization, and question answering. However, the reliance of LLMs on token-level processing (predicting one word at a time) presents some challenges. This method contrasts with human communication, which typically operates at higher levels of abstraction, such as sentences or ideas. Token-level modeling also struggles in tasks that require understanding long contexts and may produce inconsistent outputs.