Recently, Alibaba released its latest technological achievement, Animate Anyone2. This groundbreaking technology allows for the precise replacement of characters in videos. According to the case studies published on the project’s official website, users only need to provide the original video and the target character image, and the system can perform real-time character replacement while preserving the motion trajectory of the original video.
Experimental data shows that the system can completely transfer the shooting motion of soccer players to animated characters while retaining the true physical characteristics of the ball's motion trajectory. To meet the complex demands of film production, this system also supports customizable lighting adjustments and background environment adaptation, achieving a deep integration of characters and scenes.
The dynamic skeletal binding technology equipped in this system accurately tracks 28 key skeletal points of the human body, allowing for precise capture of limb movement trajectories and subtle facial expressions. In publicly tested videos, whether it's the continuous spinning motion of a dancer or the quick movements in martial arts, the replaced characters maintain continuity in their actions.
On a technical level, the system employs a shape-agnostic masking strategy, which can more effectively describe the relationship between characters and their environments. Additionally, to enhance the interaction between characters and surrounding objects, Animate Anyone2 introduces lightweight object guides. These guides are responsible for extracting features of objects that interact with the character and injecting these features into the denoising process through spatial fusion technology. This design makes the interaction between characters and their environments appear smoother and more realistic during movement.
Experimental results indicate that Animate Anyone2 excels in generating character animations that adapt to their environments. Compared to similar products currently on the market, such as Viggle and MIMO, Animate Anyone2 demonstrates higher levels of consistency, detail retention, and interaction authenticity. This marks a new height in the development of character animation technology.
With the launch of Animate Anyone2, the application potential of character animation in various fields such as gaming, film production, and virtual reality has been further expanded. It not only provides animators with more powerful tools but also offers audiences a more vivid and realistic visual experience.
Project: https://humanaigc.github.io/animate-anyone-2/
Highlights:
🌍 Animate Anyone2 achieves real interaction between characters and environments by capturing environmental features.
🎨 The new technology introduces object guides and spatial fusion to enhance the interaction effects between characters and objects.
🚀 Compared to similar technologies, Animate Anyone2 shows significant advantages in detail and consistency.
