When shopping online, have you ever been hurt by the huge difference between buyer shows and seller shows? Clearly, it's the same piece of clothing, but it looks stunning on the model, and how does it turn into something "unbearable" on you? Don't worry! The machine learning team at the University of Bielefeld in Germany has developed an AI technology called TryOffDiff that can "remove" the person in a photo, leaving only the clothing itself, and generate a standard product display image!
This technology utilizes powerful "diffusion model" AI technology, capable of identifying the shape, color, texture, and other information of the clothing from a photo, and "restoring" this information into a high-definition product display image. The generated images not only have clear and realistic details but can also automatically remove the background, just like the work of a professional photographer!
How does TryOffDiff work? Simply put, it acts like a skilled "tailor." First, it uses an image encoder called SigLIP to extract the feature information of the clothing from the photo, including color, texture, patterns, and so on, just like a tailor carefully examines the fabric. Then, it "feeds" this information to the Stable Diffusion image generation model. Stable Diffusion acts like a magical "sewing machine," capable of generating a variety of images based on the input information. Finally, Stable Diffusion generates a standard product display image based on the extracted clothing feature information, "dressing" the clothing on a virtual model, just like a tailor creating a perfect garment.
To test the effectiveness of TryOffDiff, researchers used a dataset called VITON-HD for training and testing. Experimental results showed that TryOffDiff performs exceptionally well, producing clothing images that are not only detailed and clear but also very realistic, rivaling the work of professional photographers! Compared to existing virtual fitting technologies, TryOffDiff excels in preserving clothing details, especially in patterns and logos.
The application prospects of this technology are vast; it can help consumers better understand product information and assist e-commerce platforms in enhancing product display effects, reducing return rates. In the future, when you shop for clothes online, you might only need to upload a photo of yourself to see how you look in different outfits, eliminating concerns about discrepancies between buyer shows and seller shows!
Online experience: https://huggingface.co/spaces/rizavelioglu/tryoffdiff
Project address: https://rizavelioglu.github.io/tryoffdiff/
