AI4TWINNING
Subproject: A hybrid top-down bottom-up AI method for improved digital twinning of buildings
Principle Investigator
Prof. Dr.-Ing. André Borrmann,
Chair of Computational Modeling and Simulation
Mentor
Dr.-Ing. Alexander Braun,
Chair of Computational Modeling and Simulation
Project contributor
Mansour Mehranfar, M.Sc.,
Chair of Computational Modeling and Simulation
Project summary
The project will focus on developing knowledge-driven top-down approaches for the digital twin generation. While bottom-up approaches typically start from points and successively build up surfaces and volumes, they very often face problems when it comes to creating meaningful building objects and their relations. We, therefore, apply a top-down approach by fitting a highly parametrized building model to the observed data. Based on a typology of typical office and residential buildings in Germany, we will develop a set of highly parametrized and modularized building models that provide sufficient degrees of freedom to allow the modeling of a wide range of different real-world buildings. The actual building type will be identified through methods developed by the Petzold group and through data from the Zhu project. These building models will be designed in such a way that they represent human knowledge of building design and engineering. Major parameters (the number of walls, windows, and stories is identified by a rough pattern-based interpretation of the point clouds (bottom-up). The point cloud will be provided by methods developed in the Cremers group. The actual fitting of the model to the point cloud will be approached as a high-dimensional optimization problem (top-down). For its solution, AI methods based on meta-heuristics will be investigated (such as Genetic algorithms, Simulated Annealing, Leap-Frog algorithm, etc.). The resulting digital twin of individual buildings will provide comparatively low geometric resolution, but rich semantics and consistent logic (relationships, connection points, etc,.). We will subsequently amend this rough, but a highly consistent model with local modifications by adding or removing features and objects. For the identification of these features and/or objects in the point cloud we will apply state-of-the-art ML methods on both, images and point clouds (Mask R-CNN, DeepLab, EncNet, PANet, PointNet, MVCNN, etc.) and combine them into hybrid methods. The big advantage, however, will lie in the fact that the top-down approach will allow the removal of those parts of the point clouds that can be clearly associated with major components of the buildings such as slabs, ceilings, etc. In addition, a methodology will be developed that assigns levels of confidence to individual building components based on whether and to which extent data were available to confirm its existence. Consequently, digital twins of buildings that have been captured only from the exterior will have nevertheless slabs and interior walls, however clearly marked as being an “educated guess”. For buildings where high-resolution interior scans are available, the confidence of most components will be very high. Nevertheless, hidden parts will continue to exist.
Project (preliminary) results
The team developed an automatic pipeline for creating digital twin models of the building's structure using the dense RGB point cloud. The proposed method aligns the domain knowledge with Artificial Intelligence capabilities to increase accuracy and efficiency. We proposed a novel automatic hybrid top-down, bottom-up approach for the 3D space parsing in the building environment and inferring relations between spaces. The proposed method is based on applying a deep convolutional neural network (CNN) for the semantic segmentation of main elements and the use of existing knowledge in the construction of buildings. To design highly parameterized building models, the human knowledge of building structures (walls, slabs, etc.) and their components' functional relationships are used to make them available for computational processing. We take advantage of the top-down approach by fitting a highly parametrized building model with sufficient degrees of freedom to the observed data. The model-fitting process involves solving an optimization problem using AI methods based on a hybrid Genetic Algorithm (GA)/ Nelder-Mead algorithm.
Project publications
Mehranfar, M,; Braun, A,; Borrmann, A.: A hybrid top-down, bottom-up approach for 3D space parsing using dense RGB point clouds. In: EUROPEAN CONFERENCE ON PRODUCT AND PROCESS MODELING 2022, Trondheim.
Mehranfar, M,; Braun, A,; Borrmann, A.: Automatic creation of digital building twins with rich semantics from dense RGB point clouds through semantic segmentation and model fitting. The 30th International Conference on Intelligent Computing in Engineering (EG-ICE). Abstract accepted