FunAudioLLM

Foundation model for natural voice interaction understanding and generation

CommonProductOthersSpeech RecognitionSpeech Synthesis

FunAudioLLM is a framework aimed at enhancing natural voice interaction between humans and Large Language Models (LLMs). It comprises two innovative models: SenseVoice, responsible for high-precision multi-lingual speech recognition, emotion recognition, and audio event detection; and CosyVoice, responsible for natural voice generation, supporting multi-lingual, timbre, and emotion control. SenseVoice supports over 50 languages with extremely low latency; CosyVoice excels in multi-lingual voice generation, zero-shot context generation, cross-lingual voice cloning, and instruction following capabilities. Relevant models are open-sourced on Modelscope and Huggingface, and corresponding training, inference, and fine-tuning codes are released on GitHub.

Best AI Websites & Tools

FunAudioLLM

FunAudioLLM Visit Over Time

FunAudioLLM Visit Trend

FunAudioLLM Visit Geography

FunAudioLLM Traffic Sources

FunAudioLLM Alternatives

Mini-Omni — An open-source multimodal large language model that supports real-time voice input and streaming audio output.

speech-to-speech — Open-source speech-to-speech conversion module

iFlytek Virtual Human — Full-Stack Virtual Human Multi-Scenario Application Services

EVI 2 — A new foundational voice-to-voice model that delivers a human-like conversation experience.

OpenVoiceChat — Engage in natural voice conversations with large language models.

WeST — Implement speech transcription based on LLM in just 300 lines of code.

Llama 3.1 — A cutting-edge open-source AI model that supports multilingual capabilities and advanced functionalities.

Emilia — Large-scale Multilingual Voice Generation Dataset

SenseVoice — Multilingual speech understanding model providing high-precision speech recognition and sentiment analysis.

Azure Cognitive Services Speech — Enables applications to interact intelligently through the conversion of speech to text and vice versa.

ToucanTTS — Multilingual controllable text-to-speech synthesis toolkit

sherpa-onnx — Open-source project supporting various speech recognition and speech synthesis functionalities

StreamSpeech — Real-time speech translation, bridging cross-language communication.

ChatTTS.com — Text-to-speech model for natural conversational scenarios

Cohere Aya — Multilingual AI model, supporting 101 languages.

Xunfei A.I. Intelligent Customer Service Solution — A multi-channel intelligent customer service solution based on科大讯飞speech technology.

Meta Llama 3 — Meta's new generation of open-source large language model with excellent performance

Image/Manga Translator — One-click translation of text within images.

Yi-9B — Next-generation open-source and bilingual large language model

Any GPT — A multi-modal large-scale language model

Whisper — General-purpose Speech Recognition Model

Xfyun Open Platform — An AI-powered open platform based on voice interaction

What Would They Say — Smart language assistant, making communication easier

Speechllect — Real-time AI speech-to-text/text-to-speech solution

TTSLabs — Online Voice Synthesis and Speech Recognition Service

Neon AI — Easy-to-use conversational AI, meeting the needs of businesses and families.

Youdao Translation AI Writing — Quickly generate essays, emails, and more to enhance writing efficiency.

Open Source LLM Tools — A collection of open-source large language model tools.

FreedomGPT — A platform for uncensored AI models