What is GaussianCity?
GaussianCity is an efficient, boundary-free 3D city generation framework developed by the S-Lab team at Nanyang Technological University (NTU). It is based on 3D Gaussian Splatting (3D-GS) technology and introduces a compact BEV-Point representation method, keeping the scene's VRAM usage at a constant level. This addresses the challenges of excessive memory and storage demands in large-scale scene generation.
GaussianCity introduces a spatially aware BEV-Point Decoder, which leverages point serialization and point transformation to capture both structural and contextual features of BEV points, generating high-quality 3D Gaussian attributes. The framework excels in 3D city generation from drone-view and street-view perspectives, achieving a 60x speedup compared to existing methods like CityDreamer, setting a new benchmark in both quality and efficiency.
Key Features of GaussianCity
Efficient large-scale 3D city generation: Quickly generates realistic, boundary-free city environments, supporting diverse views from drone perspectives to street-level scenes.
Low VRAM and storage requirements: The compact BEV-Point representation minimizes memory and storage demands, overcoming VRAM bottlenecks in large-scale scene generation.
High-quality visual effects: The spatially aware BEV-Point Decoder generates detailed and realistic 3D cities, supporting stylized editing and localized modifications.
Real-time rendering and interaction: Enables real-time rendering and interactive city generation, making it suitable for games, animations, and virtual reality applications.
Technical Principles of GaussianCity
3D Gaussian Splatting (3D-GS): Uses 3D Gaussian distributions to represent objects and structures in the scene, with GPU-accelerated rendering for efficient 3D scene generation.
BEV-Point Representation: Decomposes point information in 3D scenes into positional and stylistic attributes, leveraging bird's-eye view (BEV) and style lookup tables to compress data, ensuring VRAM usage remains constant regardless of scene scale.
Spatially Aware BEV-Point Decoder: Utilizes point serialization and transformation to capture structural and contextual information, generating 3D Gaussian attributes for high-quality scene rendering.
Efficient Rendering Pipeline: Combines positional encoders and modulated MLPs to generate 3D Gaussian attributes, using a Gaussian rasterizer to render the final image.
Project Links
GitHub Repository: https://github.com/hzxie/GaussianCity
Hugging Face Model Hub: https://huggingface.co/spaces/hzxie/gaussian-city
arXiv Technical Paper: https://arxiv.org/pdf/2406.06526
Applications of GaussianCity
Gaming & Virtual Reality (VR): Rapidly generates realistic virtual cities, supporting real-time rendering and interaction, reducing development costs and time.
Animation & Film Production: Efficiently generates complex urban scenes, supporting stylized rendering, improving production efficiency.
Urban Planning & Architectural Design: Quickly generates city layouts and building environments, aiding visualization in planning and design.
Autonomous Driving & Traffic Simulation: Creates realistic traffic scenarios for algorithm testing and traffic flow analysis.
Geographic Information Systems (GIS): Converts geospatial data into 3D visualizations, supporting urban development and analysis.
