Get live statistics and analysis of MrNeRF's profile on X / Twitter

Two weeks ago I fixed one of my teeth with algorithms I wrote a couple of years ago! I got hooked by 3D scanning when I started to work for a software shop in Zurich that was programming 3D computational geometry algorithms for denture scanning to produce crowns (and more). Back then, a typical reconstruction pipeline was like: scan the patient’s teeth using an intraoral scanner, reconstruct the surface mesh, design the restoration digitally, and finally mill the crown out of ceramic. We were working mostly with point clouds and meshes, but it wasn’t just math, it was craftsmanship translated into a digital process. Every micron mattered. You could literally see how a good algorithm meant a better fit in someone’s mouth. Gaussian Splatting isn’t about surface reconstruction, it’s about appearance reconstruction. It doesn’t care about explicit topology, it captures how light interacts with the scene. In a sense, it’s the opposite philosophy of the dental world: instead of modeling what the object is, it models how the object looks. 3D Gaussian Splatting enables applications like training self driving cars, teaching robots to understand their environment, creating virtual worlds, or monitoring real sites. It represents scenes as millions of small Gaussians rendered in real time without the need for meshes or textures. Coming from a world where precision geometry was everything, this shift felt natural. It’s still about reconstruction, but with a different goal: not manufacturing a perfect object, but reproducing how the world actually looks. Two weeks ago I got my first dental crown, made with the same software, reconstruction algorithms, and Swiss precision I once helped develop. I haven’t worked there in two years, but sitting in that chair and seeing the process from the other side was a proud moment. It reminded me why I love this field.

EDGS: Eliminating Densification for Efficient Convergence of 3DGS Contributions: • We show that initial triangulation based on 2D correspondences can replace the incremental refinement process, fundamentally changing how 3DGS models allocate resources. • Our method reduces the path each Gaussian must travel in parameter space. Careful initialization not only accelerates convergence but also guides optimization toward a convergence point corresponding to lower reconstruction error and thus higher reconstruction quality. • Our approach outperforms both speed-optimized and quality-focused state-of-the-art models while using only half the splats of standard 3DGS. By improving initialization rather than altering the optimization process, this method is compatible with other 3DGS acceleration techniques, making it a flexible enhancement to existing models.

FreeTimeGS: Free Gaussians at Anytime and Anywhere for Dynamic Scene Reconstruction Abstract: This paper addresses the challenge of reconstructing dynamic 3D scenes with complex motions. Recent works define 3D Gaussian primitives in the canonical space and use deformation fields to map these primitives to observation spaces, achieving real-time dynamic view synthesis. However, these methods often struggle with scenes that have complex motions due to the difficulty in optimizing deformation fields. To overcome this problem, we propose FreeTimeGS, a novel 4D representation that allows Gaussian primitives to appear at arbitrary times and locations. Unlike canonical Gaussian primitives, our representation possesses strong flexibility, improving the ability to model dynamic 3D scenes. Additionally, we endow each Gaussian primitive with a motion function, allowing it to move to neighboring regions over time, which reduces temporal redundancy. Experimental results on several datasets show that the rendering quality of our method outperforms recent methods by a large margin.

3D Gaussian Splatting Driving Simulator! You know those YouTube walking tours? I think this could become a similar trend like driving around in a splat or taking your driver's license test in a 3DGS simulator.

Europe Builds. Others Profit. 3D Gaussian Splatting (3DGS) is the perfect case study. It reflects both Europe’s brilliance and its chronic inability to turn that brilliance into business. Almost everything that made 3DGS possible was born in Europe. From the early breakthroughs in point-based rasterization in Switzerland to the cumulative research from Austria, Greece, and Germany executed in France, Europe built the foundation. No other continent can match that level of scientific collaboration and intellectual strength. The LichtFeld Studio bounty later confirmed it: the biggest performance leaps came straight out of European labs. The science was here. The innovation was here. The talent was here. But the business was not. When 3DGS exploded, my inbox filled with messages from US-based companies, not from Europe. In the United States, Luma AI and Polycam turned the paper into products within weeks. They did not wait for funding programs or EU consortia. They simply built. Then came China, which not only caught up in research but quickly outpaced everyone in commercialization. XGRID, DJI, and many others built thriving businesses around what Europe invented. Today, most 3DGS papers come from Chinese institutions rather than European ones. Meanwhile, the usual giants such as Meta, NVIDIA, Google, Netflix, and Tesla continue to iterate, integrate, and push forward. A thriving ecosystem of startups like World Labs leverages this technology to create new products and markets. The innovation cycle in the United States and China is fast, relentless, and market-driven. Europe, in contrast, remains bureaucratic and slow. We fund excellence and celebrate publications, but we rarely ship, even though some small startups are trying to change the status quo. Our researchers create the breakthroughs; others create the successful products. Until Europe finds a way to bridge the gap between laboratories and markets, it will remain the world’s research and development department: brilliant, underpaid, and underleveraged. Research is Europe’s comfort zone. Execution must become its strength. Video: One of my dynamic 3D Gaussian implementations based on the paper "Representing Long Volumetric Video with Temporal Gaussian Hierarchy."

[SIGGRAPH Asia '24 (TOG)] Representing Long Volumetric Video with Temporal Gaussian Hierarchy Contributions: • We introduce a novel, efficient, and expressive Temporal Gaussian Hierarchy representation for long volumetric video. To our knowledge, our method is the first approach capable of handling minutes of volumetric video data. • We propose a Compact Appearance Model and a new rasterization implementation to facilitate real-time, high-quality dynamic view synthesis while maintaining a compact size. • We propose a system to efficiently model long volumetric videos for the first time and demonstrate state-of-the-art dynamic view synthesis quality on the Neural3DV [Li et al. 2022], ENeRF-Outdoor [Lin et al. 2022], and MobileStage [Xu et al. 2024b] datasets, while also achieving the best rendering speed with reduced training cost and memory usage.

Human Hair Reconstruction with Strand-Aligned 3D Gaussians Contributions (cited): – We propose a new 3D line lifting scheme that uses a modified 3DGS reconstruction technique to lift 2D orientation maps into a 3D field while also providing refinement of the camera parameters; – We introduce a dual representation of hair strand polylines and 3D Gaussians to achieve differentiable rasterization of hair strands and leverage photometric constraints for strand-based hair reconstruction; – Based on these components, we propose a coarse-to-fine optimization method for prior-guided hair reconstruction that leverages both latent and explicit representations of the hairstyle.

Oh wow! I just tested Splatt3R with my own data on my computer, which creates 3D Gaussian Splats at 4 FPS on uncalibrated 512x512 2D images! It's by far the fastest 3D reconstruction method, powered by MASt3R. Check out the video!

That‘s crazy. 1.5km 3D Gaussian Driving at 280-500fps. City scale Gaussians with exciting real world applications.

MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion TL;DR: DUSt3R keeps impressing. Contributions (cited): • We introduce Motion DUSt3R (MonST3R), a geometry-first approach to dynamic scenes that directly estimates geometry in the form of pointmaps, even for moving scene elements. To this end, we identify several suitable datasets and show that, surprisingly, a small-scale fine-tuning achieves promising results for direct geometry estimation of dynamic scenes. • MonST3R obtains promising results on several downstream tasks (video depth and camera pose estimation). In particular, MonST3R offers key advantages over prior work: enhanced robustness, particularly in challenging scenarios; increased speed compared to optimization-based methods; and competitive results with specialized techniques in video depth estimation, camera pose estimation and dense reconstruction.

Europe Builds. Others Profit. 3D Gaussian Splatting (3DGS) is the perfect case study. It reflects both Europe’s brilliance and its chronic inability to turn that brilliance into business. Almost everything that made 3DGS possible was born in Europe. From the early breakthroughs in point-based rasterization in Switzerland to the cumulative research from Austria, Greece, and Germany executed in France, Europe built the foundation. No other continent can match that level of scientific collaboration and intellectual strength. The LichtFeld Studio bounty later confirmed it: the biggest performance leaps came straight out of European labs. The science was here. The innovation was here. The talent was here. But the business was not. When 3DGS exploded, my inbox filled with messages from US-based companies, not from Europe. In the United States, Luma AI and Polycam turned the paper into products within weeks. They did not wait for funding programs or EU consortia. They simply built. Then came China, which not only caught up in research but quickly outpaced everyone in commercialization. XGRID, DJI, and many others built thriving businesses around what Europe invented. Today, most 3DGS papers come from Chinese institutions rather than European ones. Meanwhile, the usual giants such as Meta, NVIDIA, Google, Netflix, and Tesla continue to iterate, integrate, and push forward. A thriving ecosystem of startups like World Labs leverages this technology to create new products and markets. The innovation cycle in the United States and China is fast, relentless, and market-driven. Europe, in contrast, remains bureaucratic and slow. We fund excellence and celebrate publications, but we rarely ship, even though some small startups are trying to change the status quo. Our researchers create the breakthroughs; others create the successful products. Until Europe finds a way to bridge the gap between laboratories and markets, it will remain the world’s research and development department: brilliant, underpaid, and underleveraged. Research is Europe’s comfort zone. Execution must become its strength. Video: One of my dynamic 3D Gaussian implementations based on the paper "Representing Long Volumetric Video with Temporal Gaussian Hierarchy."

Two weeks ago I fixed one of my teeth with algorithms I wrote a couple of years ago! I got hooked by 3D scanning when I started to work for a software shop in Zurich that was programming 3D computational geometry algorithms for denture scanning to produce crowns (and more). Back then, a typical reconstruction pipeline was like: scan the patient’s teeth using an intraoral scanner, reconstruct the surface mesh, design the restoration digitally, and finally mill the crown out of ceramic. We were working mostly with point clouds and meshes, but it wasn’t just math, it was craftsmanship translated into a digital process. Every micron mattered. You could literally see how a good algorithm meant a better fit in someone’s mouth. Gaussian Splatting isn’t about surface reconstruction, it’s about appearance reconstruction. It doesn’t care about explicit topology, it captures how light interacts with the scene. In a sense, it’s the opposite philosophy of the dental world: instead of modeling what the object is, it models how the object looks. 3D Gaussian Splatting enables applications like training self driving cars, teaching robots to understand their environment, creating virtual worlds, or monitoring real sites. It represents scenes as millions of small Gaussians rendered in real time without the need for meshes or textures. Coming from a world where precision geometry was everything, this shift felt natural. It’s still about reconstruction, but with a different goal: not manufacturing a perfect object, but reproducing how the world actually looks. Two weeks ago I got my first dental crown, made with the same software, reconstruction algorithms, and Swiss precision I once helped develop. I haven’t worked there in two years, but sitting in that chair and seeing the process from the other side was a proud moment. It reminded me why I love this field.

3D Gaussian Splatting Driving Simulator! You know those YouTube walking tours? I think this could become a similar trend like driving around in a splat or taking your driver's license test in a 3DGS simulator.

Do you think radiance field models will be killed by video diffusion before they really take off? This has got me thinking.

FreeTimeGS: Free Gaussians at Anytime and Anywhere for Dynamic Scene Reconstruction Abstract: This paper addresses the challenge of reconstructing dynamic 3D scenes with complex motions. Recent works define 3D Gaussian primitives in the canonical space and use deformation fields to map these primitives to observation spaces, achieving real-time dynamic view synthesis. However, these methods often struggle with scenes that have complex motions due to the difficulty in optimizing deformation fields. To overcome this problem, we propose FreeTimeGS, a novel 4D representation that allows Gaussian primitives to appear at arbitrary times and locations. Unlike canonical Gaussian primitives, our representation possesses strong flexibility, improving the ability to model dynamic 3D scenes. Additionally, we endow each Gaussian primitive with a motion function, allowing it to move to neighboring regions over time, which reduces temporal redundancy. Experimental results on several datasets show that the rendering quality of our method outperforms recent methods by a large margin.

Just finished a new project: forging a kitchen knife from scratch! Started with a raw metal blank, shaped, hardened, and sharpened it into a nice blade. Super satisfying and tangible! It took in total 14 hours.

Which is more crucial for 3DGS in 2025? 1. Training speed 2. Rendering quality 3. Memory efficiency 4. Dynamic (Volumetric Videos) 5. Relighting 6. Better Densification 7. ... I am curious what your take is!

Today, I spoke with the owner of the holiday resort where we are currently staying. I asked him why he hasn't created 3D models of his apartments. His simple answer: he wasn't familiar with 3DGS at all. So, I quickly demonstrated it to him using Scaniverse. He was immediately impressed and plans to show case his rooms as radiance fields. In general, I haven't seen this on any hotel website yet. It would be the ultimate marketing tool. It seems to me to be a simple multi-million-dollar business for someone to step in. Is no one daring to tap into these profits?

EDGS: Eliminating Densification for Efficient Convergence of 3DGS Contributions: • We show that initial triangulation based on 2D correspondences can replace the incremental refinement process, fundamentally changing how 3DGS models allocate resources. • Our method reduces the path each Gaussian must travel in parameter space. Careful initialization not only accelerates convergence but also guides optimization toward a convergence point corresponding to lower reconstruction error and thus higher reconstruction quality. • Our approach outperforms both speed-optimized and quality-focused state-of-the-art models while using only half the splats of standard 3DGS. By improving initialization rather than altering the optimization process, this method is compatible with other 3DGS acceleration techniques, making it a flexible enhancement to existing models.

My dynamic 3D Gaussians viewer is live, and the novel view synthesis exceeded expectations -> think SOTA quality! What's next? I'm gearing up to create a 40-second video while continually enhancing the dynamic 3GS quality. Stay tuned!

There’s a King in Town: 3D Gaussian Splatting! Let’s be honest. There’s a king ruling the world of novel view synthesis: 3D Gaussian Splatting. At this point, it doesn’t even matter if something “better” shows up. Every week, a new paper drops: Beta Splatting, Triangle Splatting, Radiant Foam, Convex Splatting. The names keep coming. Each one claims to be the next big thing. But the industry has already moved on. Meta, Apple, NVIDIA, Tesla, and a wave of smaller studios are all in on 3D Gaussian Splatting. And they are not wrong. The momentum is insane. Research output is exploding. Open-source tools are everywhere. You have Supersplat, gsplat, LichtFeldstudio, Brush, and more. Entire communities are forming around it. You can fine-tune, render, optimize, and visualize everything in a mature and well-supported and growing ecosystem. Nobody wants to throw that away. Switching to a new representation, even if it looks a bit better on paper, means starting from zero: no tools, no ecosystem, no community. 3D Gaussian Splatting has everything it needs: speed, quality, tooling, hype, and momentum. It will be hard to crown a new king.

Sora, Kling, DreamMachine, etc., and radiance fields like NeRF or Splatting in a 4D context. It doesn't take much to see where we are heading.

Human Hair Reconstruction with Strand-Aligned 3D Gaussians Contributions (cited): – We propose a new 3D line lifting scheme that uses a modified 3DGS reconstruction technique to lift 2D orientation maps into a 3D field while also providing refinement of the camera parameters; – We introduce a dual representation of hair strand polylines and 3D Gaussians to achieve differentiable rasterization of hair strands and leverage photometric constraints for strand-based hair reconstruction; – Based on these components, we propose a coarse-to-fine optimization method for prior-guided hair reconstruction that leverages both latent and explicit representations of the hairstyle.