Recently, developer scraed released LanPaint on GitHub, a zero-shot image inpainting tool. This tool aims to help users achieve high-quality image inpainting results on any Stable Diffusion (SD) model, including custom-trained models. LanPaint achieves this by iteratively prompting the model to 'think' before denoising, resulting in more seamless and accurate inpainting. A key feature of LanPaint is its zero-shot capability; users can start using it immediately without any training.

Tencent has announced the open-sourcing of its newly developed image-to-video generation framework, HunyuanVideo-I2V. This release follows the successful open-sourcing of HunyuanVideo and aims to further advancements within the open-source community. HunyuanVideo-I2V incorporates cutting-edge video generation technology, enabling the transformation of static images into dynamic video content, offering creators expanded possibilities. HunyuanVideo-I2V leverages a pre-trained...

A revolutionary technology is quietly emerging in the field of artificial intelligence. Inception Labs recently announced the Mercury series of diffusion large language models (dLLMs), a new generation of language models designed for fast and efficient high-quality text generation. Compared to traditional autoregressive large language models, Mercury boasts up to 10x faster generation speeds, achieving over 1000 tokens per second on an NVIDIA H100 GPU. This speed is...

Recently, the research team at Meta Reality Labs has jointly released an innovative generative model called 'Pippo', which can generate a dense turnaround video with a resolution of up to 1K from a randomly taken photo. This breakthrough technology marks another significant advancement in the fields of computer vision and image generation. The core of the Pippo model lies in the design of its multi-view diffusion transformer. Unlike traditional generative models, Pippo does not require any additional inputs, such as fitted parameters.

A research team from New York University, MIT, and Google has recently proposed an innovative framework aimed at addressing the bottleneck of inference time scaling in diffusion models. This groundbreaking research transcends the traditional method of simply increasing the denoising steps, opening up new avenues for improving generative model performance. The framework primarily unfolds from two dimensions: one is leveraging feedback from validators, and the second is implementing algorithms to discover better noise candidates. The research team builds on a pre-trained SiT-XL model with a resolution of 256×256, maintaining 250 fixed denoising steps.

Microsoft Research has launched a powerful AI system called MatterGen that can generate new materials with specific properties, potentially accelerating the development of key technologies such as batteries and solar panels. The introduction of MatterGen signifies a fundamental shift in how scientists discover new materials. Unlike traditional methods that screen millions of existing compounds, MatterGen directly generates new materials based on desired characteristics, akin to how AI image generators create visuals from textual descriptions.

Recently, a line art coloring method called MangaNinja has garnered widespread attention, allowing users to color target line art based on reference images with just an input of the line art and the reference image. This technology is based on a diffusion model and focuses on reference-image-guided line art coloring, significantly enhancing the accuracy of coloring and interactive control capabilities. The research team employed two innovative designs to ensure precise transfer of character details. First, they introduced a patch rearrangement module to facilitate correspondence learning between the reference color images and the target line art. Secondly, they adopted a point-driven approach.

For a long time, efficiently generating high-quality, wide-angle 3D scenes from a single image has been a challenge faced by researchers. Traditional methods often rely on multi-view data or require time-consuming per-scene optimization, and they struggle with background quality and reconstruction of unseen areas. Existing technologies, when dealing with single-view 3D scene generation, often produce errors or distortions in occluded areas due to insufficient information, along with blurry backgrounds and difficulties in inferring the geometric structure of unseen regions. While regression-based models can synthesize new views in a feedforward manner, they struggle with complex...

In the field of artificial intelligence, ByteDance's commercial technology team presents the latest achievement, the Infinity model, which has become a new leader in the auto-regressive text-to-image generation domain with its outstanding performance and innovative technology. This newly open-sourced model not only surpasses Stable Diffusion 3 in image generation quality but also demonstrates significant advantages in inference speed. The core innovation of the Infinity model lies in its use of a Bitwise Token auto-regressive framework, which predicts the next level of resolution by +1 or -1.

Recently, a research team has released a new image super resolution (SR) technology based on Diffusion Inversion, aimed at enhancing image resolution and clarity by fully utilizing the image prior information within large pre-trained diffusion models. This research was jointly completed by three scholars from different academic institutions, who aim to bring new breakthroughs to the field of image super resolution. In this technology, the researchers designed a method called 'Partial Noise Prediction'.

Are you still troubled by unclear photos? A brand new image super-resolution tool called InvSR has emerged, allowing images to be instantly transformed into clear and sharp ones with minimal processing steps. The strength of this tool lies in its utilization of the rich prior knowledge embedded in large pre-trained diffusion models, achieving efficient and high-quality image resolution enhancement. The core technology of InvSR is its innovative partial noise prediction strategy. It cleverly constructs the intermediate state of the diffusion model as the starting point for the sampling process and employs a deep noise.

With the development of digital art, automated image processing technology has received increasing attention. Recently, a research team from Tsinghua University and Tencent's ARC Lab proposed a novel image sequence coloring model called ColorFlow. This model aims to address the challenge of maintaining consistency in character and object identity while coloring black and white image sequences, fulfilling the practical needs of industries such as comics and animation. ColorFlow is a three-stage diffusion base framework that leverages contextual information by utilizing a reference image pool for black and white.

In the field of video analysis, the persistence of objects is an important cue for humans to understand that objects still exist even when completely occluded. However, current object segmentation methods mainly focus on visible (modal) objects, lacking the capability to handle non-modal (visible + invisible) objects. To address this issue, researchers proposed a two-stage method based on diffusion priors, Diffusion-Vas, aimed at improving the effectiveness of non-modal segmentation and content completion in videos, enabling the tracking of specified targets within the video and using diffusion models to complete the occluded parts.

In today's digital era, image generation technology is developing at an astonishing pace. Recently, a research team from the National University of Singapore proposed a brand-new framework — OminiControl, aimed at enhancing the flexibility and efficiency of image generation. This framework combines image conditions and fully utilizes the pre-trained Diffusion Transformer (DiT) model, providing unprecedented control capabilities. In simple terms, as long as you provide the source image, OminiControl can utilize it.

Reinforcement learning has seen significant success in recent years, but its low sample efficiency limits its application in the real world. World models, as a type of environment generation model, offer hope for addressing this issue. They can serve as simulated environments to train reinforcement learning agents with higher sample efficiency. Currently, most world models simulate environment dynamics through discrete latent variable sequences. However, this method of compressing into a compact discrete representation may overlook visual details that are critical for reinforcement learning. Meanwhile, diffusion models have become prominent in the field of image generation.