Snap demonstrated an early version of a real-time device-side image diffusion model at the Augmented World Expo, which can generate vivid augmented reality (AR) experiences. The company also introduced generative AI tools designed for AR creators.
Bobby Murphy, co-founder and CTO of Snap, stated at the event that the model is compact enough to run on smartphones and fast enough to re-render frames in real-time based on text prompts. He noted that while the emergence of generative AI image diffusion models is exciting, these models need significant speed improvements to have a major impact on augmented reality, hence Snap's team has been focused on accelerating machine learning models.
It is reported that Snapchat users will start to see lens effects based on this generative model in the coming months, with Snap planning to roll it out to creators by the end of the year. Bobby Murphy said, "This real-time device-side generative machine learning model and future models mark an exciting new direction for augmented reality, allowing us to rethink the ways we render and create AR experiences."
Additionally, Murphy announced the launch of Lens Studio 5.0, providing developers with new generative AI tools to help them create AR effects faster than ever, saving weeks or even months of time. AR creators can use the new tools to quickly generate selfie lenses, produce highly realistic machine learning facial effects in real-time, and create custom style effects that cover users' faces, bodies, and surroundings. They can also generate 3D assets and apply them to their lenses in just minutes.
Furthermore, AR creators can leverage the company's facial mesh technology to generate characters like aliens or wizards through text or image prompts, and create masks, textures, and materials in minutes. The latest version of Lens Studio also includes an AI assistant to answer questions AR creators might encounter. Snap anticipates that these new tools will significantly transform the way AR is created, enabling creators to realize their creative ideas more quickly.