SpeechGPT

Multimodal Language Model

CommonProductProgrammingSpeechMultimodal

SpeechGPT is a multimodal language model with inherent cross-modal dialogue capabilities. It can perceive and generate multimodal content and follow multimodal human instructions. SpeechGPT-Gen is an extended information chain speech generation model. SpeechAgents is a multimodal multi-agent system for human communication simulation. SpeechTokenizer is a unified speech tokenizer suitable for speech language models. The release dates and related information of these models and datasets can be found on the official website.

Best AI Websites & Tools

SpeechGPT

SpeechGPT Visit Over Time

SpeechGPT Visit Trend

SpeechGPT Visit Geography

SpeechGPT Traffic Sources

SpeechGPT Alternatives

ell — A lightweight programming library for language models, treating prompts as functions.

VideoLLaMA2-7B — A large video-language model that provides video question answering and video captioning.

Qwen-VL — General-purpose Visual Language Model

imp-v1-3b — A powerful multimodal small language model.

honeybee — Multi-modal Language Model Prediction Network

TinyGPT-V — Efficient multimodal large language model

ml-ferret — End-to-end MLLM, enabling precise referencing and localization.

DreamLLM — Multimodal Comprehension and Creation

JinaChat — More modalities, longer memory, lower cost

Pixtral-12B-2409 — A multimodal model with 12 billion parameters, integrating a visual encoder for image and text processing.

Pixtral 12B — The first multimodal Mistral model, supporting hybrid task processing for images and text.

Zhipu AI Large Model Open Platform — Integrate large models with just a few lines of code.

LLaMA-Omni — A low-latency, high-quality end-to-end speech interaction model

DataGemma — Connects large language models with Google’s data-sharing platform to reduce AI hallucination phenomena.

pixtral-12b-240910 — A multimodal large language model that supports understanding of both images and text.

West Lake AI Model — A multimodal model with high emotional and intellectual intelligence

Xincheng Lingo Voice Model — Communicating without boundaries, creating value with every conversation.

MiniCPM3-4B — High-performance third-generation MiniCPM series model.

Mini-Omni — An open-source multimodal large language model that supports real-time voice input and streaming audio output.

Zamba2-mini — A cutting-edge small language model designed for edge applications.

IMYAI Smart Assistant — A versatile smart assistant that meets diverse application needs.

Show-o — A unified transformer for multimodal understanding and generation.

mPLUG-Owl3 — A multimodal large language model that understands long image sequences.

Phi-3.5-vision — An advanced multimodal model that supports image and text understanding.

Phi-3 — An efficient and cost-effective small language model

Grok-2 — A cutting-edge language model with advanced reasoning capabilities.

LLaVA-OneVision — An efficient model for multimodal vision task transformation.

MedTrinity-25M — A large-scale multimodal medical dataset

Peach-9B-8k-Roleplay — A large language model for dialogue generation through role-playing