Recently, a research team from Carnegie Mellon University introduced a new technology called "DressRecon," aimed at reconstructing temporally consistent human models through monocular video. The impressive aspect of DressRecon is its ability to not only construct 3D models from video inputs but also to capture intricate details such as complex clothing and handheld objects.
This technology is particularly useful for scenarios involving loose clothing or interactions with handheld objects, breaking through the limitations of previous technologies. Traditional human body reconstruction often required tight clothing or multi-view calibration and data capture, and even personalized scans, making large-scale collection challenging.
The innovation of "DressRecon" lies in its combination of general human shape priors with specific video-based body deformations, allowing optimization within a single video.
The core of this technology is a neural implicit model that separates and processes body and clothing deformations, establishing motion model layers.
To capture the subtle geometric features of clothing, the research team utilized image-based prior knowledge, including human poses, surface normals, and optical flow. This information provides additional support during the optimization process, resulting in more realistic reconstructions.
DressRecon can extract high-fidelity 3D models from a single video input and can further optimize them into explicit 3D Gaussian volumes to enhance rendering quality, supporting interactive visualization.
The researchers demonstrated high-fidelity 3D reconstruction capabilities of DressRecon on challenging datasets involving complex clothing deformations and object interactions.
Additionally, the reconstructed virtual human figures can be rendered from any angle, showcasing visually impactful effects. The team compared DressRecon with multiple baseline technologies in shape reconstruction, showing that DressRecon exhibits higher fidelity in handling complex deformation structures.
Key Points:
👗 The research team introduces DressRecon technology, enabling high-quality human body reconstruction from monocular video, especially suited for scenarios with loose clothing and handheld objects.
📷 Utilizing a neural implicit model, the technology separates body and clothing deformations and leverages image-based prior knowledge to capture subtle geometric features.
🎥 The reconstruction results not only generate high-fidelity 3D models but also support rendering from any angle, enhancing the visualization experience.