describe-anything

[NeurIPS 2024] An official implementation of ShareGPT4Video: Improving Video Understanding and Generation with Better Captions

A Framework of Small-scale Large Multimodal Models

LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills

A collection of resources on applications of multi-modal learning in medical imaging.

LLaVA-Mini is a unified large multimodal model (LMM) that can support the understanding of images, high-resolution images, and videos in an efficient manner.

[CVPR 2024 Highlight] OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, llama-3.2-vision, qwen-vl, qwen2-vl, phi3-v etc.

[ECCV 2024] ShareGPT4V: Improving Large Multi-modal Models with Better Captions

Embed arbitrary modalities (images, audio, documents, etc) into large language models.

[NeurIPS 2024] This repo contains evaluation code for the paper "Are We on the Right Way for Evaluating Large Vision-Language Models"