VidSitu

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

Code for "Aligning Linguistic Words and Visual Semantic Units for Image Captioning", ACM MM 2019

遥感时序视觉语言模型：综述

[CVPR20] Video Object Grounding using Semantic Roles in Language Description (https://arxiv.org/abs/2003.10606)

A gradio based image captioning tool that uses the GPT-4-Vision API to generate detailed descriptions of images.

A Pytorch implementation of Attention on Attention module (both self and guided variants), for Visual Question Answering

Sample app to display live captioning to a WebRTC video session with the Deepgram API.

An image and video description generator using an CNN-RNN based architecture.

S2VT (seq2seq) video captioning with bahdanau & luong attention implementation in Tensorflow