Tencent has recently launched an innovative project called LVCD (Large Video Color Diffusion), a specialized tool designed for coloring animation video line art. LVCD can automatically transform black and white line art into colorful animated videos, introducing a series of breakthrough features.
Efficient Line Animation Video Coloring
LVCD is an emerging video diffusion framework aimed at achieving reference-based line animation video coloring. Unlike previous methods that relied solely on image generation models to color line drawings frame by frame, this approach utilizes large-scale pre-trained video diffusion models to generate rich-colored animated videos. This innovative method not only ensures higher temporal consistency but also more effectively handles significant motion.
Sketch-Guided ControlNet
LVCD introduces sketch-guided ControlNet technology, providing additional control for image-to-video diffusion models, allowing for better regulation of animated video generation based on line drawings. This technology significantly enhances the controllability and quality of animation generation.
Reference Attention Mechanism
LVCD also proposes a reference attention mechanism, which effectively transfers color from reference frames to other frames, especially in scenes with rapid and wide-range motion. This feature ensures color consistency in animation production, even under complex motion.
Novel Sequence Sampling Technique
During the inference phase, LVCD employs a new sequence sampling scheme, combining overlapping blending modules and previous reference attention, allowing the model to surpass the original fixed length limitations and generate long-sequence color-consistent animations. This means LVCD can handle various types of line art inputs, including hand-drawn and automatically generated line art, and produce videos of unlimited length.
Core Features
Efficient line animation video coloring based on reference frames: LVCD can utilize color information from reference frames to accurately transfer these colors to other frames.
Sketch-guided ControlNet: Enhances the controllability of animation generation.
Reference attention mechanism: Ensures color consistency in rapid motion.
Novel sequence sampling technique: Extends the temporal length of video generation.
Significantly improved frame and video quality: Suitable for complex animation production scenarios.
Experimental Results
Experimental results for LVCD show that this method significantly outperforms existing state-of-the-art techniques in terms of frame and video quality as well as temporal consistency. With high-quality, long-duration consistent animation generation, LVCD offers a new solution for animation creation, making the production of complex animations more efficient and intuitive.
The launch of this technology marks another significant step forward for Tencent in the field of animation production, promising greater convenience and creative freedom for animators in the future.
Project Link: https://luckyhzt.github.io/lvcd
