Artificial Intelligence for the automated creation of multi-scale digital twins of the built world
Borrmann, André, Prof. Dr.-Ing.
Chair of Computational Modeling and Simulation, Department of Civil and Environmental Engineering
Cremers, Daniel, Prof. Dr. rer. nat.
Chair of Computer Vision and Artificial Intelligence, Departments of Informatics and Mathematics
Kolbe, Thomas H., Prof. Dr. rer. nat.
Chair of Geoinformatics, Department of Aerospace and Geodesy
Petzold, Frank, Prof. Dr.-Ing.
Chair of Architectural Informatics, Department of Architecture
Hoegner, Ludwig, Prof. Dr.-Ing.
Photogrammetry and Remote Sensing, Department of Aerospace and Geodesy
Zhu, Xiaoxiang, Prof. Dr.-Ing.
Data Science in Earth Observation, Department of Aerospace and Geodesy
The project aims at the automated generation of a system of interrelated digital twins of the built environment spanning multiple resolution scales providing rich semantics and coherent geometry. To this end, we will research a multi-scale, multi-sensor, multi-method approach combining terrestrial, airborne, and spaceborne acquisition, different sensors (visible, thermal, LiDAR, Radar) and different processing methods integrating top-down and bottom-up AI approaches. The key concept of this proposal providing a ground-breaking step forward lies in deriving building information and intelligently fusing the resulting information by AI-based methods, thus closing information gaps and increasing completeness, accuracy and reliance of the resulting digital twins. To facilitate the process and improve the results, we make extensive use of informed machine learning by exploiting explicit knowledge on the design and construction of built facilities. The goal of the project is not to create a single monolithic digital twin, but instead a system of interlinked twins across different scales, providing the opportunity to seamlessly integrate city, district and building models as well as keep these up-to-date and consistent. As testbed and demonstration scenario, we will use the inner city of Munich, a zone around the central TUM campus where large data sets from various sensors are already available.
AI4TWINNING consists of 6 subprojects
- AI and Earth Observation for a Multi-scale 3D Reconstruction of the Built Environment
- Thermal 3D mapping and CNN analysis
- AI-based / knowledge-driven reconstruction of semantic building data using multiple image data sources
- Digital twin data consistency across multiple representations
- Knowledge-driven 3D Reconstruction from Camera Streams
- A hybrid top-down bottom-up AI method for improved digital twinning of buildings