New Technology for Coloring Line Art MangaNinja: Input Line Art and Reference Images for Accurate Coloring

AIbase基地

Published inAI News · 4 min read · Jan 16, 2025

144

Recently, a line art coloring method called MangaNinja has attracted widespread attention. By simply inputting a line drawing and a reference image, it can color the target line art based on the reference image. This technology is based on a diffusion model and focuses on reference image-guided line art coloring, significantly enhancing the accuracy of coloring and interactive control.

The research team ensured the precise transmission of character details through two innovative designs. First, they introduced a patch rearrangement module to facilitate correspondence learning between the reference color image and the target line art. Second, they adopted a point-driven control scheme, allowing users to finely match colors.

In their experiments, the researchers constructed a self-collected benchmark dataset and compared it with existing coloring methods. The results showed that MangaNinja significantly outperformed other methods in coloring accuracy and generated image quality. An important feature of this method is that it can achieve high-quality coloring results without relying on point guidance.

MangaNinja has demonstrated its unique advantages when dealing with challenging scenes. For instance, when facing significant character pose variations or missing details, point guidance can help resolve these issues. Additionally, when multiple objects are involved, point guidance effectively prevents color confusion. Furthermore, users can perform multi-reference image coloring by selecting specific areas from multiple reference images, providing guidance for various elements of the line art and effectively addressing conflicts between similar visual elements.

This technology also supports semantic color matching and fine control when using different reference images. The researchers believe that this interactive coloring approach can help users find inspiration during the coloring process and provide more creative possibilities.

Project: https://johanan528.github.io/MangaNinjia/

GitHub: https://github.com/ali-vilab/MangaNinjia

Key Points:

🌟 MangaNinja is a reference image-based line art coloring method with precise matching and detailed control capabilities.

🎨 Through an innovative patch rearrangement module and point-driven control scheme, MangaNinja significantly improves coloring accuracy and image quality.

🖌️ This technology can tackle diverse coloring challenges, including extreme poses and coordination of multiple reference images, achieving a high-quality interactive coloring experience.

Google AI Introduces a Fundamental Framework for Scaling Inference Time in Diffusion Models

A research team from New York University, MIT, and Google has recently proposed an innovative framework aimed at addressing the bottleneck of inference time scaling in diffusion models. This groundbreaking research transcends the traditional method of simply increasing the denoising steps, opening up new avenues for improving generative model performance. The framework primarily unfolds from two dimensions: one is leveraging feedback from validators, and the second is implementing algorithms to discover better noise candidates. The research team builds on a pre-trained SiT-XL model with a resolution of 256×256, maintaining 250 fixed denoising steps.

Microsoft Launches Innovative AI Model MatterGen to Create New Materials Based on Specific Needs

Microsoft Research has launched a powerful AI system called MatterGen that can generate new materials with specific properties, potentially accelerating the development of key technologies such as batteries and solar panels. The introduction of MatterGen signifies a fundamental shift in how scientists discover new materials. Unlike traditional methods that screen millions of existing compounds, MatterGen directly generates new materials based on desired characteristics, akin to how AI image generators create visuals from textual descriptions.

Single Image Transforms into 3D Wonderland: Efficiently Building High-Quality 3D Scenes with Wonderland

For a long time, efficiently generating high-quality, wide-angle 3D scenes from a single image has been a challenge faced by researchers. Traditional methods often rely on multi-view data or require time-consuming per-scene optimization, and they struggle with background quality and reconstruction of unseen areas. Existing technologies, when dealing with single-view 3D scene generation, often produce errors or distortions in occluded areas due to insufficient information, along with blurry backgrounds and difficulties in inferring the geometric structure of unseen regions. While regression-based models can synthesize new views in a feedforward manner, they struggle with complex...

ByteDance Releases Infinity: A New Breakthrough in Auto-Regressive Text-to-Image Generation, Outperforming Diffusion Models

In the field of artificial intelligence, ByteDance's commercial technology team presents the latest achievement, the Infinity model, which has become a new leader in the auto-regressive text-to-image generation domain with its outstanding performance and innovative technology. This newly open-sourced model not only surpasses Stable Diffusion 3 in image generation quality but also demonstrates significant advantages in inference speed. The core innovation of the Infinity model lies in its use of a Bitwise Token auto-regressive framework, which predicts the next level of resolution by +1 or -1.