Real-time radiance field rendering that achieves state-of-the-art visual quality while maintaining competitive training times and enabling high-quality real-time novel-view synthesis at 1080p resolution.
Overview
A breakthrough representation that preserves the desirable properties of continuous volumetric radiance fields while avoiding unnecessary computation in empty space.
Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates.
We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (≥ 30 fps) novel-view synthesis at 1080p resolution.
Core Elements
01
Starting from sparse points produced during camera calibration, the scene is represented with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields while avoiding unnecessary computation in empty space.
02
Performs interleaved optimization and density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene.
03
A fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows real-time rendering at 1080p resolution.
04
Demonstrates state-of-the-art visual quality and real-time rendering on several established datasets including MipNeRF360, Tanks & Temples, and Deep Blending.
Pipeline
A complete pipeline from sparse SfM points to real-time rendering at 1080p.
The pipeline starts with sparse point clouds from SfM (Structure from Motion) calibration. These points serve as the initial seed for the 3D Gaussian representation.
Each input point initializes a 3D Gaussian with position, anisotropic covariance, opacity, and spherical harmonics coefficients for view-dependent appearance.
Through interleaved optimization and adaptive density control, Gaussians are added, removed, and refined to accurately capture scene geometry and appearance.
A differentiable tile-based rasterizer efficiently sorts and renders Gaussians, enabling fast alpha-blending and real-time frame rates at 1080p resolution.
Why It Matters
Achieves ≥ 30 fps novel-view synthesis at 1080p resolution for unbounded, complete scenes.
Training times are comparable to or faster than NeRF-based methods while achieving superior quality.
Uses explicit 3D Gaussians instead of neural networks, making rendering fast and interpretable.
Quick Start
The codebase assumes a CUDA-ready GPU, Conda environment, and compatible C++ compiler. If that matches your setup, this is the shortest path to a first run.
Clone with submodules to get the optimizer, viewers, and utility scripts.
git clone https://github.com/graphdeco-inria/gaussian-splatting --recursive
cd gaussian-splatting
Install all dependencies including PyTorch and CUDA extensions in a dedicated environment.
conda env create --file environment.yml
conda activate gaussian_splatting
Prepare your images with COLMAP and run the optimizer to produce a 3D Gaussian model.
python train.py -s <path to COLMAP or NeRF Synthetic dataset>
Use the SIBR real-time viewer to render and explore your trained model interactively.
./<SIBR install dir>/bin/SIBR_gaussianViewer_app -m <path to trained model>
New Features
Integrated drop-in replacements for a 1.6x speedup using the default optimizer and a 2.7x speedup using the sparse Adam optimizer.
Uses depth maps as priors during optimization for better reconstruction on untextured surfaces, reducing floaters.
Optimizes affine transformations per image to compensate for exposure changes in uncontrolled captures.
EWA filtering from Mip-Splatting removes aliasing artifacts. OpenXR support enables VR viewing of Gaussian scenes.
FAQ
Start here if you want to understand the hardware requirements, data pipelines, or common setup issues without reading the whole repository first.
Who created 3D Gaussian Splatting?
3D Gaussian Splatting was created by Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis from Inria at the GRAPHDECO research group. The research was funded by the ERC Advanced grant FUNGRAPH No 788065.
What hardware do I need to train a model?
A CUDA-ready GPU with Compute Capability 7.0+ and 24 GB of VRAM to train to paper evaluation quality. For smaller VRAM configurations, see recommendations on adjusting the densification parameters in the FAQ.
Can I run this on Windows, Linux, or macOS?
The codebase has been tested on Windows 10 and Ubuntu Linux 22.04. The viewers require OpenGL 4.5. macOS and other platforms are not officially supported. A Colab template is available for quick access.
What datasets are available for evaluation?
Standard evaluation uses MipNeRF360, Tanks & Temples, and Deep Blending datasets. Pre-trained models (14 GB), evaluation images (7 GB), and COLMAP SfM datasets are available from the project page.
How can I process my own images?
Use the provided convert.py script with COLMAP and optionally ImageMagick. Place your images in an input directory and run the conversion pipeline to generate undistorted, resized images with SfM information.
What's the BibTeX citation for this work?
@Article{kerbl3Dgaussians,
author = {Kerbl, Bernhard and Kopanas, Georgios and Leimk{\"u}hler, Thomas and Drettakis, George},
title = {3D Gaussian Splatting for Real-Time Radiance Field Rendering},
journal = {ACM Transactions on Graphics},
number = {4},
volume = {42},
month = {July},
year = {2023},
url = {https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/}
}
Primary Sources
Explore the official paper, codebase, and supplementary materials for complete details.
The source for the paper, datasets, pre-trained models, and SIBR viewer binaries.
Official authors implementation with PyTorch optimizer, network viewer, and real-time viewer.
Published in ACM Transactions on Graphics (SIGGRAPH 2023), detailing the method and results.
Conference presentation and supplementary video showcasing real-time rendering results.