VidSitu

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

Code for "Aligning Linguistic Words and Visual Semantic Units for Image Captioning", ACM MM 2019

[CVPR20] Video Object Grounding using Semantic Roles in Language Description (https://arxiv.org/abs/2003.10606)

A gradio based image captioning tool that uses the GPT-4-Vision API to generate detailed descriptions of images.

A Pytorch implementation of Attention on Attention module (both self and guided variants), for Visual Question Answering

Sample app to display live captioning to a WebRTC video session with the Deepgram API.

An image and video description generator using an CNN-RNN based architecture.

S2VT (seq2seq) video captioning with bahdanau & luong attention implementation in Tensorflow

Automated reddit scraper and video creator