In the wave of building the digital virtual world, how to transform portraits into high-fidelity 3D characters has always been a hot spot of technological exploration. Now, with the emergence of RodinHD technology, it is possible to generate high-fidelity 3D avatar models based on portraits, even including hair details.
Product Access: https://top.aibase.com/tool/rodinhd
The core of RodinHD technology lies in its innovative three-plane fitting and generation framework. In the fitting stage, the technology customizes a high-resolution three-plane for each character and equips it with a shared decoder to render realistic images. In the generation stage, it generates richly detailed high-resolution three-planes by learning the cascading of basic and upsampling diffusion models. During this process, the conditional portrait images are injected in a hierarchical manner, providing strong support for the presentation of the details of the 3D avatar.
During the continuous fitting of three-planes for multiple characters, the decoder may forget the knowledge of previous characters and over-adapt to new characters. To address this issue, RodinHD introduces data scheduling strategies such as task replay and weight merging regularization, effectively enhancing the decoder's ability to capture and present the details of new characters.
In terms of high-resolution three-plane diffusion, RodinHD optimizes noise scheduling, making the 32-channel three-plane more stable even at the same noise level, reducing the loss of details. Moreover, to extract more details from portrait images, RodinHD utilizes a pre-trained variational autoencoder for the calculation of multi-scale feature representations, ensuring the complete retention of basic visual details.
The breakthrough of RodinHD technology not only lies in its capture of complex details like hairstyles but also in its comprehensive innovation of existing 3D avatar generation technology. After optimizing training with 46,000 avatars, the 3D characters generated by RodinHD exceed any previous method in detail and can adapt to various wild portrait inputs.
Conditional avatar generation from synthetic portraits
Creation from wild portrait avatars
Avatar creation under text conditions
Key points:
🛠️ **Three-plane fitting and generation**: RodinHD customizes high-resolution three-planes and a shared decoder for each character through two stages - fitting and generation.
🔄 **Overcoming catastrophic forgetting**: Through task replay and weight merging regularization, RodinHD effectively solves the problem of decoder forgetting in continuous fitting.
🎨 **High-resolution three-plane diffusion**: Optimized noise scheduling and multi-scale feature representation enable RodinHD to achieve an unprecedented level of detail in the presentation of 3D characters.
