Do you often hesitate when shopping for clothes online, worried that your favorite pieces might not fit once you bring them home? Fear not, because a groundbreaking technology is here to save the day! Google Research's latest development, Fashion-VDM, allows you to experience the thrill of trying on various beautiful outfits from the comfort of your own home!
So, what exactly is this magical Fashion-VDM? In simple terms, it's a video diffusion model. Just provide a photo of the garment and a video of yourself, and it can generate a video of you wearing that piece, with incredibly realistic results!
You might wonder, aren't there already virtual try-on softwares available? Most of those are image-based, generating only static pictures, and the results are often appalling—the clothes look like stickers on you, devoid of any realism. Fashion-VDM, however, is entirely different. It creates dynamic videos, showcasing the garment's appearance from various angles and even simulating dynamic changes like folds and sway, just like a real try-on experience!
The secret behind Fashion-VDM lies in its use of split-CFG (Classifier-Free Guidance Split) technology, which allows for more precise control over the information of the person and the clothing, ensuring that the generated video retains your personal features while perfectly showcasing the garment's details.
To make the videos smoother and more natural, Fashion-VDM employs a progressive temporal training strategy. It initially trains the model with a large amount of image data, then gradually increases the duration of video data training, ultimately producing ultra-long videos of up to 64 frames, eliminating any stuttering or flickering!
Even more impressive, Fashion-VDM combines image and video data for joint training, meaning it not only learns the details of the clothing from images but also the movements of the person and the dynamic changes of the clothing from videos, resulting in more authentic and convincing try-on videos.
To test the effectiveness of Fashion-VDM, researchers compared it with other virtual try-on and animation technologies on the market. Fashion-VDM emerged victorious in terms of image quality, video smoothness, and garment fidelity!
Of course, Fashion-VDM still has some limitations, such as potentially inaccurate details when handling occluded garment areas and slight deformations in body shape. However, with continuous technological advancements, Fashion-VDM is expected to become increasingly perfect, ultimately revolutionizing our online shopping experience!
No more worries about buying the wrong clothes online. With Fashion-VDM, traditional fitting rooms are simply outmatched!
Project link: https://johannakarras.github.io/Fashion-VDM/