Simulation optimization of highway hard shoulder running based on multi-agent deep deterministic policy gradient algorithm - Hu, Lipeng; Tang, Jinjun; Zou, Guoqing; Li, Zhitao; Zeng, Jie; Li, Mingyang (Alexandria engineering journal, 2025)
To alleviate traffic congestion and reduce vehicle emissions, the use of hard shoulder running (HSR) has emerged as a sustainable and cost-effective active traffic management technology. However, optimizing the utilization of HSR remains a critical challenge for improving highway traffic congestion. To tackle this issue, the Multi-Agent Deep Deterministic Policy Gradient with spatio-temporal constraints (STC-MADDPG) algorithm based on multi-agent reinforcement learning is proposed in this paper. To verify the effectiveness of the proposed algorithm, the present study utilizes a Simulation of Urban Mobility (SUMO) platform to construct a simulation environment. The optimal HSR strategy is then determined for four different service levels of highways. Additionally, the granularity of control is adjusted by varying the number of agents, allowing for a comprehensive analysis and evaluation of the varying effectiveness of different control levels across different service levels. Through in-depth investigation into the two strategies under the fourth service level, it is discovered that fewer sections each agent controls yields better results when congestion becomes more severe. The experimental results clearly demonstrate the superiority of the optimized strategy for HSR using the STC-MADDPG algorithm, compared to the “no open” strategy. Specifically, the maximum reductions achieved in terms of total vehicle travel time, Time Integrated Time-to-collision, CO emissions, CO2 emissions, and NOx emissions are 37.4 %, 34.1 %, 28.0 %, 17.1 %, and 27.2 % respectively. This comprehensive evaluation of the algorithm's effectiveness covers three key aspects: driving efficiency, driving safety, and environmental protection. The findings conclusively demonstrate the positive impact of the proposed algorithm on all three fronts. •We propose a spatio-temporal MARL algorithm to optimize HSR strategy.•The method’s effectiveness in efficiency, emissions, and safety across four levels is assessed using the SUMO.•An extended study compares strategies with 4 and 8 agents at service level 4 under severe congestion.•The results of the evaluation confirmed the validity of the proposed methodology through actual case testing.