Recently, Jige Technology proposed a new framework called DriveDreamer4D, aimed at leveraging prior knowledge from world models to improve the reconstruction quality of 4D driving scenes. Traditional 4D scene reconstruction methods primarily rely on two main approaches: NeRF and 3DGS. NeRF is like a super painter, capable of rendering a collection of photos into a 3D model using neural networks. On the other hand, 3DGS simulates various objects in the scene using a set of three-dimensional Gaussian functions. However, both methods have their limitations.

The research team proposed a simpler approach, IllumiNeRF: first, a diffusion model under specific lighting conditions is used to relight each input image, and then these relighted images are used to reconstruct a Neural Radiance Field (NeRF) to present new perspectives under the target lighting. They demonstrated that this strategy is surprisingly competitive and achieved state-of-the-art results in several relighting benchmarks.