Condition-based maintenance for redundant systems considering spare inventory with stochastic lead time - Wang, Jun; Fu, Yuqiang; Zhou, Jian; Yang, Lechang; Yang, Yating (Reliability engineering & system safety, 2025)
•Stochastic maintenance time and spare ordering lead time are considered.•An improved Q-learning algorithm is proposed with convergence proved.•The effectiveness of the proposed algorithm is shown by numerical examples.•Simulated environment based on discrete event simulation method is established. Condition-based maintenance (CBM) and spare provisioning are both important to guarantee the operation of redundant systems composed of degrading components. However, most existing studies on joint optimization of CBM and spare inventory assume maintenance actions are instantaneous and the lead time for spares are fixed, which are not consistent with the reality. Therefore, this paper focus on the joint optimization problems to minimize the total cost rate considering stochastic maintenance time for components and stochastic lead time for spares. The problem is modeled as a Markov decision process model and solved by an improved reinforcement learning algorithm, i.e., the improved Q-learning algorithm, which converges more quickly and reaches a smaller value of the total cost rate than the traditional Q-learning algorithm. Moreover, the simulated environment based on discrete event simulation method is introduced in detail and the convergence of the algorithm is proved theoretically. Based on the numerical study, we further demonstrate the convergence and effectiveness of the proposed algorithm and perform sensitivity analysis on several model parameters to provide management insights for decision makers.