DreamMesh4D
A novel framework for generating high-quality 4D objects from monocular videos.
CommonProductImage3D Generation4D Objects
DreamMesh4D is a novel framework that combines mesh representation with sparse control deformation techniques to generate high-quality 4D objects from monocular videos. This technology addresses the challenges of spatial-temporal consistency and surface texture quality seen in traditional methods by integrating implicit neural radiance fields (NeRF) or explicit Gaussian drawing as underlying representations. Drawing inspiration from modern 3D animation workflows, DreamMesh4D binds Gaussian drawing to triangle mesh surfaces, enabling differentiable optimization of textures and mesh vertices. The framework starts with a rough mesh provided by single-image 3D generation methods and constructs a deformation graph by uniformly sampling sparse points to enhance computational efficiency while providing additional constraints. Through two-stage learning, it leverages reference view photometric loss, score distillation loss, and other regularization losses to effectively learn static surface Gaussians, mesh vertices, and dynamic deformation networks. DreamMesh4D outperforms previous video-to-4D generation methods in rendering quality and spatial-temporal consistency, and its mesh-based representation is compatible with modern geometric processes, showcasing its potential in the 3D gaming and film industries.