Popular GitHub Repositories for Video Understanding

A curated list of action recognition and related area resources

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

An open-source toolbox for action understanding based on PyTorch

[CVPR 2024 Highlight?] Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding

[CVPR 2023] VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking

Temporal Segments LSTM and Temporal-Inception for Activity Recognition

Source code for "Taming Visually Guided Sound Generation" (Oral at the BMVC 2021)

? ? ? [NeurIPS 2024] Hawk: Learning to Understand Open-World Video Anomalies

This is the official implementation of our paper "Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension"

Awesome OVD-OVS - A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Future

PyTorch implementation of BEVT (CVPR 2022) https://arxiv.org/abs/2112.01529

[NeurIPS 2022] Zero-Shot Video Question Answering via Frozen Bidirectional Language Models

Easily convert RGB video data (e.g. .avi) to the TensorFlow tfrecords file format for training e.g. a NN in TensorFlow. This implementation allows to limit the number of frames per video to be stored in the tfrecords.

[ECCV 2024?] Official implementation of the paper "ST-LLM: Large Language Models Are Effective Temporal Learners"

Official pytorch repository for CG-DETR "Correlation-guided Query-Dependency Calibration in Video Representation Learning for Temporal Grounding"

Useful Toolbox for Anomaly Detection

[ECCV 2022] Official Pytorch Implementation of the paper : " Zero-Shot Temporal Action Detection via Vision-Language Prompting "

[NeurIPS 2022 Spotlight] VideoMAE for Action Detection

FreeVA: Offline MLLM as Training-Free Video Assistant

Source code for "Synchformer: Efficient Synchronization from Sparse Cues" (ICASSP 2024)