Recently, the research team at Alibaba Group has introduced a novel framework named UniPortrait, which is dedicated to personalized processing of portrait images, achieving single-character consistency, multi-character consistency, and style referencing.
This framework is not only capable of handling images of a single identity but also excels in high-quality personalized customization in multi-character scenarios. UniPortrait is characterized by its ability to maintain highly realistic facial features and supports a wide range of facial editing functions. Users can even generate the images they desire using free-form text descriptions without the need for fixed layouts.
Single-Character Consistency
The core of UniPortrait consists of two modules: the ID Embedding Module and the ID Routing Module. The ID Embedding Module is responsible for extracting editable facial features for each identity and embedding these features in a decoupled manner into the context space of the diffusion model. Subsequently, the ID Routing Module adaptively combines and allocates these features based on different regions in image synthesis, thereby achieving personalized customization for both single and multiple identities.
Multi-Character Consistency
During the training process, UniPortrait employs a meticulously designed two-stage training scheme. The first stage focuses on training with a single identity, while the second stage involves fine-tuning for multiple identities. Through this training approach, UniPortrait outperforms existing methods in both single and multiple identity customization, and experimental results also demonstrate its good scalability, being generally compatible with existing generation control tools.
The introduction of UniPortrait brings new possibilities for personalized customization of portrait images, especially in the areas of free-form prompts and diverse layout generation. The research team has showcased numerous personalized examples of both single and multiple identities, highlighting the significant potential of this framework in practical applications. In summary, UniPortrait not only enhances the quality of image generation but also paves the way for various future application scenarios.
Product Project Entry:
Try It Out: https://huggingface.co/spaces/Junjie96/UniPortrait
Key Points:
🌟 UniPortrait is a groundbreaking framework focused on personalized image processing for both single and multiple identities, featuring high-quality facial feature retention.
✍️ The framework comprises the ID Embedding Module and the ID Routing Module, achieving efficient customization through a two-stage training scheme.
🚀 UniPortrait offers rich possibilities for portrait personalization, supporting free-form text descriptions and diverse layout generation.