It is observed that the presence of budget constraints forces the agents to cluster their actions in specific system parts, autonomously emphasizing their prioritization to certain components of the system. For example, it is notable that repair and replacement resources are primarily designated to components 3,4,8,9 which are the components with the most aggressive deterioration model. In the low budget case, inspections are mainly conducted for component 3, which is the component whose functionality is more likely to cause failure of the most reliable path of the system. Overall, it is noted that the value of information provided by inspections fades in lower budget scenarios, since the agents prefer to reserve insection resources for major interventions in an event of disruption (e.g. link failure).

​

The structure of the system is shown below (from Type I to III, we move from a less to a more aggressive deterioration model).  A realization of a 50-year risk-constrained policy is also shown.

Besides the apparent diversification of different component policies, a notable pattern in this risk-constrained case is that the agents develop opportunistic strategies when failure events occur in components of the same link, whereas, similarly, intervention actions, which in general cause partial or total link closures, are synchronized for same-link components. The reason why the agents develop this behavior is that maintenance-induced network disruptions are thereby minimized. Again, this behavior, similarly to the observed re-prioritization of inspection and maintenance resources under budget constraints shown previously, is autonomously learned by the agents without any user-defined enforcement or implicit reward-based penalization/motivation.

References:​

​

Andriotis, C.P., and Papakonstantinou, K.G., “Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints”, Reliability Engineering & System Safety (under review), arXiv preprint arXiv:2007.01380, 2020. [Link]

​

Andriotis, C.P., and Papakonstantinou, K.G., “Managing engineering systems with large state and action spaces through deep reinforcement learning”, Reliability Engineering & System Safety, 191 (11), 106483, 2019. [Link]

​

Andriotis C.P., and Papakonstantinou, K.G., “Life-cycle policies for large engineering systems under complete and partial observability”, 13th International Conference on Applications of Statistics and Probability in Civil Engineering (ICASP), Seoul, South Korea, 2019. [Link]

​​​​​​​​​

​Resources:

​

Data, Documentation, Presentations