MIRROR

An open source implementation of CLIP.

Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration

The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".

A concise but complete implementation of CLIP with various experimental improvements from recent papers

A curated list of Visual Question Answering(VQA)(Image/Video Question Answering),Visual Question Generation ,Visual Dialog ,Visual Commonsense Reasoning and related area.

[CVPR 2024 & NeurIPS 2024] EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI

Build high-performance AI models with modular building blocks

Library for Digital Pathology Image Processing

Pathology Language and Image Pre-Training (PLIP) is the first vision and language foundation model for Pathology AI (Nature Medicine). PLIP is a large-scale pre-trained model that can be used to extract visual and language features from pathology images and text description. The model is a fine-tuned version of the original CLIP model.

[CVPR 2024] Official PyTorch Code for "PromptKD: Unsupervised Prompt Distillation for Vision-Language Models"