AI4TWINNING
Subproject: Knowledge-driven 3D Reconstruction from Camera Streams
Principle Investigator
Prof. Dr. rer. nat. Daniel Cremers,
Chair of Computer Vision and Artificial Intelligence, Departments of Informatics and Mathematics
Project contributor
Sergei Solonets, M.Eng.,
Chair of Computer Vision and Artificial Intelligence, Departments of Informatics and Mathematics
Project summary/aims
Develop a knowledge-driven approach to 3D reconstruction:
The first aim of the project is to develop a model-based approach to 3D reconstruction, which can leverage prior knowledge about the scene by decomposing it to primitives. As the world consists of the repeated structures, representation of such structures can be learned independently on different scenes.
The second aim of the project is to provide more accurate reconstruction using prior knowledge about the objects on the scene. Appart from the naturally dynamic objects, such as humans or vehicles, there are also contextually dynamic objects, which could move or be stationary depending on the situation. This is especially relevant to the area of BIM.
Project (preliminary) results
Developed an application to construct a reconstruction of a scene using small blobs of structure (Nerfels). The current work aims to use this formulation to learn similar structures in different scenes (primitives mining).
Formulated a closed-form solution for feature-metric alignment, which can be used for camera-based reconstruction in the partially dynamic scene. (publication pending)