Nexa AI recently unveiled its new OmniAudio-2.6B audio language model, designed to meet the efficient deployment demands of edge devices. Unlike traditional architectures that separate automatic speech recognition (ASR) and language models, OmniAudio-2.6B integrates Gemma-2-2b, Whisper Turbo, and a custom projector into a unified framework. This design eliminates the inefficiencies and delays associated with linking various components in traditional systems, making it especially suitable for resource-constrained computing.

The American startup Useful Sensors has launched an open-source speech recognition model called Moonshine. Moonshine is designed to process audio data more efficiently, using computational resources more economically and achieving processing speeds five times faster than OpenAI's Whisper. This new model is specifically built for real-time applications on resource-constrained hardware and features a flexible architecture. Unlike Whisper, which processes audio in fixed 30-second segments, Moonshine offers a different approach.

Moore Threads has announced the open-source release of its audio understanding model, MooER, making it the first large-scale open-source speech model based on domestically produced full-feature GPUs. MooER supports Chinese and English speech recognition and translation, utilizing a three-part model structure that demonstrates robust multilingual processing capabilities. The inference code and a model trained on 5000 hours of data have been released as open source, with plans to further open-source training code and an enhanced version trained on 80,000 hours of data. In comparative testing, MooER-5K has shown outstanding performance, achieving a Chinese CER of 4.21% and an English WER of 17.98%, particularly.

At the Volcano Engine AI Innovation Roadshow in Shanghai on August 21, 2024, Volcano Engine showcased a comprehensive upgrade of the Doubao Large Model. This includes the Doubao Text-to-Image Model, which has improved text and image matching capabilities for long texts, the Doubao Speech Recognition Model, which reduced error rates by up to 40% across multiple public test sets, and upgrades to the Doubao Speech Synthesis Model, enhancing streaming speech synthesis abilities for real-time responses and accurate punctuation. Volcano Engine also released a real-time interactive solution for Conversational AI, integrating the Doubao Large Model with real-time audio and video technology, providing end-to-end capabilities.

The Seed-ASR engine launched by ByteDance achieves high-precision recognition of Mandarin, 13 Chinese dialects, and 7 foreign languages through massive training data, significantly enhancing the convenience of cross-language communication. Its key advantage lies in its excellent contextual awareness, accurately recognizing proper nouns, place names, and keywords by incorporating historical information, especially performing exceptionally well in specific scenarios, thereby improving recognition accuracy. Whether in daily conversations, complex meetings, or interactions among multiple people in noisy environments, Seed-ASR can transcribe accurately. It can also recognize various professional terms.

Linly-Dubbing is an open-source AI tool designed specifically for multilingual dubbing and translation of videos, simplifying the video localization process. Users just need to upload their videos, and the tool will automatically download, translate, dub, and generate subtitles in multiple languages. Its powerful AI recognition and translation capabilities, in collaboration with models like GPT, ensure high quality and natural-sounding translations. Notably, its voice cloning technology can generate voices that closely resemble the original dubbing, ensuring lip-syncing and enhancing the viewing experience. This tool is ideal for international education and global entertainment.

Alibaba recently launched the new open-source voice model Qwen2-Audio, which excels in speech recognition, translation, and audio analysis, achieving significant performance improvements. Qwen2-Audio offers a basic version and an instruction fine-tuning version, supporting multiple languages such as Chinese, Cantonese, French, English, and Japanese, facilitating sentiment analysis and translation applications. Compared to Qwen-Audio, Qwen2-Audio features comprehensive optimizations in architecture and performance, utilizing more natural language prompts during the pre-training phase.

Israeli AI company aiOla has released an open source speech recognition model named Whisper Medusa, which is based on an improved architecture design that incorporates multi-head attention mechanisms, allowing it to process speech 50% faster than OpenAI's Whisper model. Whisper Medusa makes parallel predictions of ten tokens instead of the traditional one at a time, significantly enhancing speech recognition speed while maintaining performance. Its innovative training method employs weak supervision, freezing the backbone system and utilizing...

Israeli AI startup aiOla has launched an open-source speech recognition model called Whisper-Medusa, which is 50% faster than OpenAI's Whisper. The model is built on Whisper, utilizing an innovative 'multi-head attention' architecture that allows for the prediction of more tokens at once, resulting in a significant speed increase while maintaining accuracy. Whisper-Medusa has been released on Hugging Face under the MIT License, permitting both research and commercial use.

A recent report from a Nature sub-journal highlights an AI system built using living human brain cells, which improved speech recognition accuracy from 30%-40% to 78%. The system can perform unsupervised learning, akin to neural networks, and its learning ability is related to the connections of neural cells in brain organoids. Composed of 'mini-brains' and microelectrodes, significant improvements were achieved after two days of training, but it can only recognize the speaker without understanding the content. While performance is expected to improve, brain organoids can only survive for 1-2 months, and external devices have high power consumption. This research indicates that truly general biological computing systems may take decades to develop.

Amazon has released a next-generation ASR system that covers over 100 languages, providing comprehensive automatic speech recognition services. The speech foundation model improves accuracy by 20% to 50%, with enhancements of 30% to 70% in challenging areas such as telephone speech. The system supports multiple features, including automatic punctuation, custom vocabulary, automatic language identification, and speaker separation. Thousands of businesses are leveraging Amazon Transcribe to unlock insights from audio content, enhancing accessibility and discoverability.

{1:Meta has released the world's largest open-source multimodal translation model, supporting 100 languages and capable of recognizing regional dialects. 2:SeamlessM4T is the first integrated AI translation model that supports multimodal translation tasks including speech-to-text, speech-to-speech, text-to-speech, and text-to-text. 3:SeamlessM4T integrates previously released translation models by Meta, such as NLLB and MMS, using 270,000 hours of aligned speech and text data.}