Dynamic multi-tour order picking in an automotive-part warehouse based on attention-aware deep reinforcement learning
Uloženo v:
| Název: | Dynamic multi-tour order picking in an automotive-part warehouse based on attention-aware deep reinforcement learning |
|---|---|
| Autoři: | Wang, Xiaohan, Zhang, Lin, Wang, Lihui, Ruiz Zúñiga, Enrique, Wang, Xi Vincent, Flores-García, Erik |
| Informace o vydavateli: | School of Automation Science and Electrical Engineering, Beihang University, Beijing, China ; Department of Production Engineering, KTH Royal Institute of Technology, Stockholm, Sweden School of Automation Science and Electrical Engineering, Beihang University, Beijing, China ; State Key Laboratory of Intelligent Manufacturing System Technology, Beijing, China Department of Production Engineering, KTH Royal Institute of Technology, Stockholm, Sweden Department of Sustainable Production Development, KTH Royal Institute of Technology, Stockholm, Sweden Department of Production Engineering, KTH Royal Institute of Technology, Stockholm, Sweden Department of Production Engineering, KTH Royal Institute of Technology, Stockholm, Sweden 2025 |
| Druh dokumentu: | Electronic Resource |
| Abstrakt: | Dynamic order picking has usually demonstrated significant impacts on production efficiency in warehouse management. In the context of an automotive-part warehouse, this paper addresses a dynamic multi-tour order-picking problem based on a novel attention-aware deep reinforcement learning-based (ADRL) method. The multi-tour represents that one order-picking task must be split into multiple tours due to the cart capacity and the operator’s workload constraints. First, the multi-tour order-picking problem is formulated as a mathematical model, and then reformulated as a Markov decision process. Second, a novel DRL-based method is proposed to solve it effectively. Compared to the existing DRL-based methods, this approach employs multi-head attention to perceive warehouse situations. Additionally, three improvements are proposed to further strengthen the solution quality and generalization, including (1) the extra location representation to align the batch length during training, (2) the dynamic decoding to integrate real-time information of the warehouse environment during inference, and (3) the proximal policy optimization with entropy bonus to facilitate action exploration during training. Finally, comparison experiments based on thousands of order-picking instances from the Swedish warehouse validated that the proposed ADRL could outperform the other twelve DRL-based methods at most by 40.6%, considering the optimization objective. Furthermore, the performance gap between ADRL and seven evolutionary algorithms is controlled within 3%, while ADRL can be hundreds or thousands of times faster than these EAs regarding the solving speed. © 2025 Published by Elsevier Ltd.Corresponding author at: School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, ChinaThe authors would like to acknowledge the support of Swedish Innovation Agency (VINNOVA). This study is part of the Dynamic Scheduling of Assembly and Logistics Systems using AI (Dynamic SALSA) project. This research is also supported by the National Key R&D Program of China (No. 2023YFB3308201). Dynamic Scheduling of Assembly and Logistics Systems using AI (Dynamic SALSA) |
| Témata: | Smart manufacturing system, Industry 5.0, Manual order picking, Deep reinforcement learning, Intelligent decision-making, Computer Sciences, Datavetenskap (datalogi), Article in journal, info:eu-repo/semantics/article, text |
| DOI: | 10.1016.j.rcim.2025.102959 |
| URL: | Robotics and Computer-Integrated Manufacturing, 0736-5845, 2025, 94 |
| Dostupnost: | Open access content. Open access content info:eu-repo/semantics/restrictedAccess |
| Poznámka: | English |
| Other Numbers: | UPE oai:DiVA.org:his-24924 0000-0001-8679-8049 0000-0003-4180-6003 0000-0001-9694-0483 doi:10.1016/j.rcim.2025.102959 ISI:001401135400001 Scopus 2-s2.0-85214875132 1512096063 |
| Přispívající zdroj: | UPPSALA UNIV LIBR From OAIster®, provided by the OCLC Cooperative. |
| Přístupové číslo: | edsoai.on1512096063 |
| Databáze: | OAIster |
Nájsť tento článok vo Web of Science | Abstrakt: | Dynamic order picking has usually demonstrated significant impacts on production efficiency in warehouse management. In the context of an automotive-part warehouse, this paper addresses a dynamic multi-tour order-picking problem based on a novel attention-aware deep reinforcement learning-based (ADRL) method. The multi-tour represents that one order-picking task must be split into multiple tours due to the cart capacity and the operator’s workload constraints. First, the multi-tour order-picking problem is formulated as a mathematical model, and then reformulated as a Markov decision process. Second, a novel DRL-based method is proposed to solve it effectively. Compared to the existing DRL-based methods, this approach employs multi-head attention to perceive warehouse situations. Additionally, three improvements are proposed to further strengthen the solution quality and generalization, including (1) the extra location representation to align the batch length during training, (2) the dynamic decoding to integrate real-time information of the warehouse environment during inference, and (3) the proximal policy optimization with entropy bonus to facilitate action exploration during training. Finally, comparison experiments based on thousands of order-picking instances from the Swedish warehouse validated that the proposed ADRL could outperform the other twelve DRL-based methods at most by 40.6%, considering the optimization objective. Furthermore, the performance gap between ADRL and seven evolutionary algorithms is controlled within 3%, while ADRL can be hundreds or thousands of times faster than these EAs regarding the solving speed.<br />© 2025 Published by Elsevier Ltd.Corresponding author at: School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, ChinaThe authors would like to acknowledge the support of Swedish Innovation Agency (VINNOVA). This study is part of the Dynamic Scheduling of Assembly and Logistics Systems using AI (Dynamic SALSA) project. This research is also supported by the National Key R&D Program of China (No. 2023YFB3308201).<br />Dynamic Scheduling of Assembly and Logistics Systems using AI (Dynamic SALSA) |
|---|---|
| DOI: | 10.1016.j.rcim.2025.102959 |