Time‐Varying 3D Gaussian Splatting Representation for Dynamic Scenes

Abstract

ABSTRACT Real‐time photorealistic novel view synthesis of dynamic scenes remains a challenging task, primarily due to the inherent complexity of temporal dynamics and motion patterns. Despite recent methods based on Gaussian Splatting having shown considerable progress in this regard, they are still limited by high memory consumption. In this paper, we propose a time‐varying 3D Gaussian splatting (TVGS) representation for dynamic scenes, which incorporates two key components. First, we model the scene using 3D Gaussians endowed with temporal opacity and time‐varying motion parameters. These attributes effectively capture transient phenomena such as the sudden appearance or disappearance of dynamic elements. Second, we introduce an adaptive density control mechanism to optimise the distribution of these time‐varying 3D Gaussians throughout the sequence. As an explicit dynamic scene representation, TVGS not only achieves high‐fidelity view synthesis but also attains a real‐time rendering speed of 160 FPS on the Neural 3D Video Dataset using a single RTX 4090 GPU.