Recently, the LivePortrait human portrait video generation framework, open-sourced by the Kuaishou Kelin team, has caused a sensation in the field of artificial intelligence. This innovative tool can generate lifelike dynamic videos from a single static image, showcasing the immense potential of AI technology in the video generation sector.
Upon its release, LivePortrait garnered widespread attention in the open-source community. In a short span, the project has amassed 7.5K stars on GitHub, making it one of the hottest AI projects currently. Moreover, it attracted the personal experience of Thomas Wolf, Chief Strategy Officer at HuggingFace, and topped the trends on the HuggingFace platform among all applications.
The core advantage of LivePortrait lies in its remarkable ability to "transplant" expressions. It not only allows characters in static images to blink, smile, or turn their heads but also accurately replicates expressions and dynamics from one person to another, regardless of style, suitable for various styles such as realistic, oil painting, sculpture, and 3D rendering.
The application scope of this tool is extremely broad. From individual portraits to family photos, from people to pets, LivePortrait can achieve vivid dynamic effects. More impressively, it can precisely control expressions in videos, such as adjusting the curvature of the corners of the mouth or the size of the eyes, providing creators with unprecedented control over expressions.
Technically, LivePortrait employs an innovative framework based on implicit keypoints, differing from the current mainstream diffusion model-based methods. It significantly enhances the model's generalization, expressiveness, and texture quality through a two-stage training process. The first stage focuses on framework improvements, including high-quality data collation, mixed training, network architecture upgrades, etc. The second stage improves the precision of facial expression details through the training of fitting modules and redirection modules.
Compared to existing methods, LivePortrait excels in generation quality and driving accuracy, especially in capturing subtle expressions and maintaining original image textures. Although it slightly lags behind diffusion model-based methods in some aspects, LivePortrait boasts high inference efficiency, with a frame generation speed of 12.8 milliseconds on an RTX4090 GPU, significantly surpassing existing diffusion model methods.
The emergence of LivePortrait not only showcases the latest advancements of AI technology in video generation but also brings new possibilities to the creative industry. As this technology continues to evolve, we can expect more astonishing AI video generation applications to emerge in the near future, providing richer and more convenient creation tools for content creators and ordinary users.
Project link: https://top.aibase.com/tool/liveportrait
LivePortrait experience link:
https://huggingface.co/spaces/KwaiVGI/LivePortrait