MultimodalTransformers

An open source implementation of CLIP.

Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration

The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".

A concise but complete implementation of CLIP with various experimental improvements from recent papers

A curated list of Visual Question Answering(VQA)(Image/Video Question Answering),Visual Question Generation ,Visual Dialog ,Visual Commonsense Reasoning and related area.

Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm

[CVPR 2024 & NeurIPS 2024] EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI

Build high-performance AI models with modular building blocks

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"

[NeurIPS2023] Parameter-efficient Tuning of Large-scale Multimodal Foundation Model