NVLM 1.0

Cutting-edge multimodal large language model

CommonProductProductivityMultimodalLarge Language Model

NVLM 1.0 is part of the cutting-edge series of multimodal large language models launched by NVIDIA ADLR, achieving industry-leading performance in visual-language tasks, comparable to top proprietary and open-access models. This model improves accuracy in pure text tasks following multimodal training. The open-source model weights and Megatron-Core training code offer valuable resources for the community.

Achieves industry-leading performance in visual-language tasks.
Improves accuracy in pure text tasks after multimodal training.
Provides open-source model weights and training code for community use and research.
Achieves top scores in benchmarks such as OCRBench and VQAv2.
Demonstrates exceptional instruction-following abilities and image captioning capabilities in multimodal tasks.
Understands humor behind images
performs OCR text label recognition
and reasons about humor.
Executes mathematical reasoning and coding based on visual information.

\NVLM 1.0 is designed for researchers and developers who need to process large amounts of visual and linguistic data
particularly in the fields of machine learning
artificial intelligence
and data science. It aids users in achieving breakthroughs in image recognition
natural language processing

Used for image captioning
improving accuracy in understanding image content.
Provides step-by-step mathematical reasoning in problem-solving for mathematics and programming.
Employed for OCR tasks
recognizing and processing text within images.

Visit the official NVIDIA ADLR website to download the model weights and training code for NVLM 1.0.
Read the documentation to understand the model architecture and usage.
Fine-tune the model as needed to adapt it to specific visual-language tasks.
Train the model using the Megatron-Core training code.
Utilize the model for tasks such as image captioning

Visit

NVLM 1.0 Visit Over Time

Monthly Visits

267073

Bounce Rate

65.74%

Page per Visit

2.0

Visit Duration

00:01:08

NVLM 1.0 Visit Trend

NVLM 1.0 Visit Geography

NVLM 1.0 Traffic Sources

NVLM 1.0 Alternatives

InternVL 2.5 — Open-source multimodal large language model series

Productivity•multimodal•large language model

InternVL2_5-4B — A multimodal large language model that integrates visual and language understanding.

Image•Multimodal•Large Language Model

InternVL2_5-2B — A multimodal large language model that supports deep interaction between images and text.

Image•Multimodal•Large Language Model

InternVL2_5-1B — A large multimodal language model that supports image and text understanding.

Image•Multimodal•Large Language Model

InternVL2_5-8B — A multimodal large language model supporting interaction understanding between images and text.

Image•Multimodal•Large Language Model

InternVL2_5-26B — A large multimodal language model that integrates visual and linguistic understanding.

Image•Multimodal•Large Language Model

InternVL2_5-78B — Advanced multimodal large language model series

Image•Multimodal•Large Language Model

OLMo 2 7B — A large language model with 7 billion parameters, enhancing natural language processing capabilities.

Programming•Large Language Model•Natural Language Processing

Qwen2.5-Coder-1.5B-Instruct-GGUF — The Qwen2.5-Coder series features a 1.5B parameter instruction-tuned model.

Programming•Code Generation•Code Reasoning

Pixtral-Large-Instruct-2411 — A 124B-parameter multimodal large language model.

Productivity•Multimodal•Large Language Model

102

Ferret-UI-Llama8b — A multimodal large language model based on Llama-3-8B, focused on UI tasks.

Programming•Multimodal•Large Language Model

186

Open-O1 — An open-source large language model matching proprietary capabilities.

Programming•Open Source•Large Language Model

126

Llama-3.2-11B-Vision — A multimodal large language model that supports image and text processing.

Productivity•Multimodal•Image Processing

684

Reflection Llama-3.1 70B — The world's leading open-source large language model

Productivity•Large Language Model•Natural Language Processing

294

OLMoE-1B-7B — An efficient open-source large language model.

Productivity•Large Language Model•Open Source

Mini-Omni — An open-source multimodal large language model that supports real-time voice input and streaming audio output.

Productivity•Multimodal•Speech Recognition

456

mPLUG-Owl3 — A multimodal large language model that understands long image sequences.

Image•Multimodal•Image Understanding

144

nanoPerplexityAI — The simplest open-source implementation

Programming•Search Engine•Large Language Model

114

lmms-finetune — A unified codebase for fine-tuning large multimodal models.

Programming•Fine-tuning•Multimodal

Llama 3.1 — A cutting-edge open-source AI model that supports multilingual capabilities and advanced functionalities.

2760

OmAgent — A multimodal intelligent agent framework for tackling complex tasks

Programming•Multimodal•Intelligent Agent

162

Meta Llama 3 — Meta's new generation of open-source large language model with excellent performance

GlobalTrending•Large Model•Open Source

4632

MA-LMM — MA-LMM is a large-scale multimodal model for long-term video understanding.

Video•Video Understanding•Multimodal

684

mPLUG-DocOwl — A modular multimodal large language model for document understanding

Productivity•Document Understanding•Multimodal

234

MiniGemini — A multimodal large language model capable of understanding and generating images

Programming•Multimodal•Visual Language Model

2406

Infini-Megrez — Multimodal understanding model for edge applications, enabling intelligent edge solutions through hardware-software collaboration.

Productivity•Artificial Intelligence•Deep Learning

Best AI Websites & Tools

NVLM 1.0

NVLM 1.0 Visit Over Time

NVLM 1.0 Visit Trend

NVLM 1.0 Visit Geography

NVLM 1.0 Traffic Sources

NVLM 1.0 Alternatives

InternVL 2.5 — Open-source multimodal large language model series

InternVL2_5-4B — A multimodal large language model that integrates visual and language understanding.

InternVL2_5-2B — A multimodal large language model that supports deep interaction between images and text.

InternVL2_5-1B — A large multimodal language model that supports image and text understanding.

InternVL2_5-8B — A multimodal large language model supporting interaction understanding between images and text.

InternVL2_5-26B — A large multimodal language model that integrates visual and linguistic understanding.

InternVL2_5-78B — Advanced multimodal large language model series

OLMo 2 7B — A large language model with 7 billion parameters, enhancing natural language processing capabilities.

Qwen2.5-Coder-1.5B-Instruct-GGUF — The Qwen2.5-Coder series features a 1.5B parameter instruction-tuned model.

Pixtral-Large-Instruct-2411 — A 124B-parameter multimodal large language model.

Ferret-UI-Llama8b — A multimodal large language model based on Llama-3-8B, focused on UI tasks.

Open-O1 — An open-source large language model matching proprietary capabilities.

Llama-3.2-11B-Vision — A multimodal large language model that supports image and text processing.

Reflection Llama-3.1 70B — The world's leading open-source large language model

OLMoE-1B-7B — An efficient open-source large language model.

Mini-Omni — An open-source multimodal large language model that supports real-time voice input and streaming audio output.

mPLUG-Owl3 — A multimodal large language model that understands long image sequences.

nanoPerplexityAI — The simplest open-source implementation

lmms-finetune — A unified codebase for fine-tuning large multimodal models.

Llama 3.1 — A cutting-edge open-source AI model that supports multilingual capabilities and advanced functionalities.

OmAgent — A multimodal intelligent agent framework for tackling complex tasks

Meta Llama 3 — Meta's new generation of open-source large language model with excellent performance

MA-LMM — MA-LMM is a large-scale multimodal model for long-term video understanding.

mPLUG-DocOwl — A modular multimodal large language model for document understanding

MiniGemini — A multimodal large language model capable of understanding and generating images

M2UGen — Multimodal Music Understanding and Generation System

ChatHi — Multi-functional AI Intelligent Assistant Platform

Unitree RL GYM — Unitree robot platform for reinforcement learning

Infini-Megrez — Multimodal understanding model for edge applications, enabling intelligent edge solutions through hardware-software collaboration.