Prompt-The-Missing

A curated list of different papers and datasets in various areas of audio-visual processing

Multimodal Prompting with Missing Modalities for Visual Recognition, CVPR'23

This repo contains the official PyTorch implementation of: Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptation

Source code for "Synchformer: Efficient Synchronization from Sparse Cues" (ICASSP 2024)

Source code for "Sparse in Space and Time: Audio-visual Synchronisation with Trainable Selectors." (Spotlight at the BMVC 2022)

A comprehensive review of techniques to address the missing-modality problem for medical images

[Paper][ISWC 2023] Rethinking Uncertainly Missing and Ambiguous Visual Modality in Multi-Modal Entity Alignment

Official source code of the INTERSPEECH 2023 paper: "Audio-Visual Speech Separation in Noisy Environments with a Lightweight Iterative Model" (AVLIT)

Code for InterSpeech 2024 Paper: LipGER: Visually-Conditioned Generative Error Correction for Robust Automatic Speech Recognition

Code and datasets for 'Move2Hear: Active Audio-Visual Source Separation' (ICCV 2021)