JoyHallo

A digital avatar model supporting Mandarin video production.

PremiumNewProductVideoArtificial IntelligenceVideo Generation
JoyHallo is a digital avatar model designed specifically for Mandarin video generation. It has created the jdh-Hallo dataset by collecting 29 hours of Mandarin video from employees of JD Health International Co., Ltd. This dataset covers a variety of ages and speaking styles, including conversational and specialized medical topics. The JoyHallo model utilizes a Chinese wav2vec2 model for audio feature embedding and introduces a semi-decoupled structure to capture the relationships between lip movements, expressions, and postures, improving information utilization efficiency and accelerating inference speed by 14.3%. Additionally, JoyHallo demonstrates excellent performance in generating English videos, showcasing outstanding cross-language generation capabilities.
Visit

JoyHallo Alternatives