Popular GitHub Repositories for Vision Language Pretraining

DeepSeek-VL: Towards Real-World Vision-Language Understanding

[EMNLP 2023 Demo] Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

[ACL 2024 ?] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.

Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm

Official Repository of paper VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding

Multi-Aspect Vision Language Pretraining - CVPR2024

This repository provides a comprehensive collection of research papers focused on multimodal representation learning, all of which have been cited and discussed in the survey just accepted https://dl.acm.org/doi/abs/10.1145/3617833 .

Easy wrapper for inserting LoRA layers in CLIP.

Demographic Bias of Vision-Language Foundation Models in Medical Imaging

Code for ACL 2023 Oral Paper: ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning

Unofficial implementation for Sigmoid Loss for Language Image Pre-Training