Popular GitHub Repositories for Mllm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

Reasoning in LLMs: Papers and Resources, including Chain-of-Thought, OpenAI o1, and DeepSeek-R1 ?

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

? Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos

??Woodpecker: Hallucination Correction for Multimodal Large Language Models

OpenEMMA, a permissively licensed open source "reproduction" of Waymo’s EMMA model.

[ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization

Custom ComfyUI nodes for Vision Language Models, Large Language Models, Image to Music, Text to Music, Consistent and Random Creative Prompt Generation

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train your own 8B/14B LLaVA-training-like MLLM on RTX3090/4090 24GB.

This project is the official implementation of 'LLMGA: Multimodal Large Language Model based Generation Assistant', ECCV2024 Oral

Awesome_Multimodel is a curated GitHub repository that provides a comprehensive collection of resources for Multimodal Large Language Models (MLLM). It covers datasets, tuning techniques, in-context learning, visual reasoning, foundational models, and more. Stay updated with the latest advancement.

??Long-VITA: Scaling Large Multi-modal Models to 1 Million Tokens with Leading Short-Context Accuracy

Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning

mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video (ICML 2023)

Official code for Paper "Mantis: Multi-Image Instruction Tuning" [TMLR2024]

AUITestAgent is the first automatic, natural language-driven GUI testing tool for mobile apps, capable of fully automating the entire process of GUI interaction and function verification.

A CPU Realtime VLM in 500M. Surpassed Moondream2 and SmolVLM. Training from scratch with ease.

A RLHF Infrastructure for Vision-Language Models

[CVPR2024] Generative Region-Language Pretraining for Open-Ended Object Detection

R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization