What is 3D Gaussian Splatting (3DGS)?
Real-time viewing a place or object with freedom of movement goes beyond what 2D images offer.
3D Gaussian Splatting (3DGS) Emerges
Before understanding why it works, why is 3DGS so exciting? What if you could walk into a photo and look around in 3D space? - reminiscent of the Black Mirror episode called Eulogy starring Paul Giamatti.
Reconstructing a real-world space in 3D locks a moment in time with a level of immersive detail not found from a flat image. The CG world has been developing this for decades. There a various ways to do this - photogrammetry, using a point-cloud, sculpting a 3D mesh, radiance fields. It is a time and cost intensive process - whether points or polygons are used.
The 3DGS "splat" renders a realistic 3D scene in real time in a lightweight size that loads on a website! 3DGS pairs fast real-time rendering with fast data capture that happens at a walk-around pace. And it captures glass reflections.
Authors and creators have done a very good job at explaining this technology so I direct you to those who helped me understand just how cool this new tech is for everyday people:
"On the most basic level, they are a way of representing an image as tiny, elliptical blobs that you can stretch or squish. You also give each blob a color, change how see-through it is, and blend it with neighboring splats until they form a continuous surface. When a machine-learning algorithm does this for millions of splats per image, they come together to create 3D scenes much more realistic than any other form of 3D imagery." - Kirsten M. Johnson
Source: https://nianticlabs.com/news/splats-change-everything?hl=en
Source: Created by makinnovations.grprojects, kiriengine.app
The History of 3DGS
A 3DGS radiance field doesn't use a triangular mesh or a voxel grid like traditional 3D scanning. Instead, the entire world is reconstructed from millions of independent, semi-transparent ellipsoids, often called "splats."
1990s: Early concepts of "Light Fields" and point-based 3D rendering are theorized by computer scientists, laying the groundwork for capturing how light travels through a physical space.
2020: Neural Radiance Fields (NeRF) become a popular approach for photorealistic 3D reconstruction. NeRFs prove that AI can stitch regular 2D photos into incredible 3D scenes, but they are slow to process and require massive computing power.
2023: A breakthrough scientific paper from Inria titled "3D Gaussian Splatting for Real-Time Radiance Field Rendering" is released and wins the highly coveted Best Paper award at SIGGRAPH 2023. It effectively solves the speed problem of NeRFs, allowing for real-time, ultra-high-quality 3D environments to be generated and viewed instantaneously.
References
Source: https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
Source: https://nianticlabs.com/news/splats-change-everything?hl=en
Source: https://medium.com/data-science/a-comprehensive-overview-of-gaussian-splatting-e7d570081362
Source:
Source:
Source:
Source:
