INFRA.RELEARN
Intelligent infrastructure maintenance with deep reinforcement learning
Principal Investigators
Prof. Dr. Daniel Straub,
Engineering Risk Analysis Group, Departments of Civil and Environmental Engineering
Prof. Dr.-Ing. habil. Alois Christian Knoll,
Chair of Robotics, Artificial Intelligence and Real-time Systems, Department of Informatics
Project contributor
Carmen Buliga, M.Sc.,
Engineering Risk Analysis Group, Departments of Civil and Environmental Engineering
Daniel Hettegger, M.Sc.,
Chair of Robotics, Artificial Intelligence and Real-time Systems, Department of Informatics
Motivation and Goals Optimal planning of maintenance of civil infrastructure is challenging because these systems are large and include many interactions and constraints. In addition, there is significant uncertainty on the condition of the infrastructure components. This project aims at utilizing and tailoring deep reinforcement learning (DRL) for identifying optimal maintenance strategies. Special focus is given to the explainability of strategies and sim-2-real gaps.
Recent Results
- GNNs improve the performance of reinforcement learning for planning in networks that change over time (Hettegger et al. in prep)
- Theory-guided reinforcement learning leads to better performance in out-of-distribution applications (Koutas et al. subm)
- Extending a foundation model for time-series forcasting (Lag Llama) to improve the remaining-useful-life prediction for engineering systems
- We initiated an international group (with colleages at TU Delft, Penn State, ETH) to set up an RL competition on optimal maintenance of road infrastructure to be proposed for NeurIPS 2025.
Selected documentation
Published papers | Koutas D., Bismut E., Straub D. (2024). An investigation of belief-free DRL and MCTS for inspection and maintenance planning. Journal of Infrastructure Preservation and Resilience, 5(6). Hettegger D., Buliga C., Bismut E., Straub D., Knoll A. (2023). Investigation of Inspection and Maintenance Optimization with Deep Reinforcement Learning in Absence of Belief States. Proc. 14th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP14, Dublin, Ireland. |
Submitted papers | Koutas D., Hettegger D., Papakonstantinuou K., Straub D.: Convex is back: Solving Belief MDPs via Convexity-informed Deep Reinforcement Learning. Proc. 12th International Conference on Learning Representations (ICLR 2024), under review. |
In preparation | Probabilistic Multivariate Time Series Forecasting: Extending Lag-Llama to Multivariate Time Series IMP-act: Realistic Road Inspection and Maintenance Challenge using DRL. |