Artificial Intelligence for the automated creation of multi-scale digital twins of the built world

Principal Investigators 

  • Borrmann, André, Prof. Dr.-Ing.
    Chair of Computational Modeling and Simulation, Department of Civil and Environmental Engineering

  • Cremers, Daniel, Prof. Dr. rer. nat.
    Chair of Computer Vision and Artificial Intelligence, Departments of Informatics and Mathematics

  • Kolbe, Thomas H., Prof. Dr. rer. nat.
    Chair of Geoinformatics, Department of Aerospace and Geodesy

  • Petzold, Frank, Prof. Dr.-Ing.
    Chair of Architectural Informatics, Department of Architecture

  • Hoegner, Ludwig, Prof. Dr.-Ing.
    Photogrammetry and Remote Sensing, Department of Aerospace and Geodesy

  • Zhu, Xiaoxiang, Prof. Dr.-Ing.
    Data Science in Earth Observation, Department of Aerospace and Geodesy


The project aims at the automated generation of a system of interrelated digital twins of the built environment spanning multiple resolution scales providing rich semantics and coherent geometry. To this end, we will research a multi-scale, multi-sensor, multi-method approach combining terrestrial, airborne, and spaceborne acquisition, different sensors (visible, thermal, LiDAR, Radar) and different processing methods integrating top-down and bottom-up AI approaches. The key concept of this proposal providing a ground-breaking step forward lies in deriving building information and intelligently fusing the resulting information by AI-based methods, thus closing information gaps and increasing completeness, accuracy and reliance of the resulting digital twins. To facilitate the process and improve the results, we make extensive use of informed machine learning by exploiting explicit knowledge on the design and construction of built facilities. The goal of the project is not to create a single monolithic digital twin, but instead a system of interlinked twins across different scales, providing the opportunity to seamlessly integrate city, district and building models as well as keep these up-to-date and consistent. As testbed and demonstration scenario, we will use the inner city of Munich, a zone around the central TUM campus where large data sets from various sensors are already available.