1. Why is it difficult to truly understand existing buildings digitally?

Many modern 3D methods can generate impressively realistic digital models. But there’s a big difference between visual reconstruction and structural understanding.
A building isn’t just a collection of surfaces. It consists of volumes, spatial relationships, layers, and geometric regularities. Yet many current methods model scenes implicitly - they reproduce how something looks, but not necessarily how it is built.
Within the context of the USP focus topic, this gap is particularly interesting: How can digital methods help not only visualize existing structures, but also capture them in a meaningful way?

2. What motivated you to develop LinPrim and what problem were you trying to solve?

Many powerful 3D reconstruction methods today rely on continuous fields or point-based representations. These are highly flexible, but often difficult to interpret and only partially compatible with classical geometric workflows.
The motivation behind LinPrim was therefore: Can we develop a representation that is both differentiable and optimizable? That way it could be trained with modern ML methods, while still having an explicit geometric structure.
We wanted a representation that stays close to classical volumetric or mesh-based concepts without giving up the advantages of data‑driven optimization.

3. In simple terms: How does LinPrim work?

LinPrim doesn’t model a scene as a continuous field, but as a collection of small volumetric building blocks, so‑called linear primitives, such as tetrahedra or octahedra.
Each of these elements has clearly defined faces, an explicit volume, and parameterizable properties like color or density. These primitives are optimized to produce a consistent 3D reconstruction from the input images. A specially designed differentiable rasterizer allows us to compute gradients and optimize the entire system end‑to‑end.
The result is a scene composed of explicit geometric elements, not just an abstract field, but a structured volumetric representation.

4. What makes this approach stand out compared to other methods?

Many modern approaches prioritize visual quality and efficiency. LinPrim adds a structural objective. Because it uses explicit volumetric primitives, the resulting representation is more geometrically interpretable, closer to classical CAD or mesh models, and potentially more suitable for structural analysis.
For applications in construction planning or building assessment, this explicitness is particularly valuable. When the internal representation already consists of well‑defined volumetric elements, the step toward structural interpretation becomes much shorter.

5. What worked especially well in the project, and what turned out to be more challenging than expected?

It was fascinating to see how strongly the choice of representation influences the behavior of the entire system. Small design decisions in the geometry of the primitives had major effects on stability and quality.
The most challenging part was developing an efficient, differentiable rasterizer that is both numerically stable and scalable to larger scenes. This is where mathematics, GPU implementation, and optimization intersect very tightly.
Projects like this highlight how important it is not only to design new methods, but also to question the fundamental representations we build them on.

Please contact Nicolas von Lützow for further information.