FunAudioLLM

Foundation model for natural voice interaction understanding and generation

CommonProductOthersSpeech RecognitionSpeech Synthesis

FunAudioLLM is a framework aimed at enhancing natural voice interaction between humans and Large Language Models (LLMs). It comprises two innovative models: SenseVoice, responsible for high-precision multi-lingual speech recognition, emotion recognition, and audio event detection; and CosyVoice, responsible for natural voice generation, supporting multi-lingual, timbre, and emotion control. SenseVoice supports over 50 languages with extremely low latency; CosyVoice excels in multi-lingual voice generation, zero-shot context generation, cross-lingual voice cloning, and instruction following capabilities. Relevant models are open-sourced on Modelscope and Huggingface, and corresponding training, inference, and fine-tuning codes are released on GitHub.

Best AI Websites & Tools

FunAudioLLM

FunAudioLLM Visit Over Time

FunAudioLLM Visit Trend

FunAudioLLM Visit Geography

FunAudioLLM Traffic Sources

FunAudioLLM Alternatives

Mini-Omni — An open-source multimodal large language model that supports real-time voice input and streaming audio output.

speech-to-speech — Open-source speech-to-speech conversion module

Transcribro — Private, on-device speech recognition keyboard and text service on the Android platform.

GLM-4-Voice — An end-to-end English-Chinese voice dialogue model.

Reverb — Open-source inference code for speech recognition and speaker segmentation models.

OmniSenseVoice — Ultra-fast speech recognition with precise timestamps

Deepgram Voice Agent API — Real-time conversational AI with one-click API integration.

iFlytek Virtual Human — Full-Stack Virtual Human Multi-Scenario Application Services

EVI 2 — A new foundational voice-to-voice model that delivers a human-like conversation experience.

OpenVoiceChat — Engage in natural voice conversations with large language models.

WeST — Implement speech transcription based on LLM in just 300 lines of code.

Llama 3.1 — A cutting-edge open-source AI model that supports multilingual capabilities and advanced functionalities.

Emilia — Large-scale Multilingual Voice Generation Dataset

SenseVoice — Multilingual speech understanding model providing high-precision speech recognition and sentiment analysis.

Azure Cognitive Services Speech — Enables applications to interact intelligently through the conversion of speech to text and vice versa.

ToucanTTS — Multilingual controllable text-to-speech synthesis toolkit

sherpa-onnx — Open-source project supporting various speech recognition and speech synthesis functionalities

StreamSpeech — Real-time speech translation, bridging cross-language communication.

ChatTTS.com — Text-to-speech model for natural conversational scenarios

Cohere Aya — Multilingual AI model, supporting 101 languages.

Xunfei A.I. Intelligent Customer Service Solution — A multi-channel intelligent customer service solution based on科大讯飞speech technology.

Meta Llama 3 — Meta's new generation of open-source large language model with excellent performance

Image/Manga Translator — One-click translation of text within images.

Yi-9B — Next-generation open-source and bilingual large language model

Any GPT — A multi-modal large-scale language model

Whisper — General-purpose Speech Recognition Model

Xfyun Open Platform — An AI-powered open platform based on voice interaction

What Would They Say — Smart language assistant, making communication easier

Speechllect — Real-time AI speech-to-text/text-to-speech solution