Multi-Agent Deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks

As urban traffic condition is diverse and complicated, applying reinforcement learning to reduce traffic congestion becomes one of the hot and promising topics. Especially, how to coordinate the traffic light controllers of multiple intersections is a key challenge for multi-agent reinforcement lear...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on vehicular technology Vol. 69; no. 8; pp. 8243 - 8256
Main Authors: Wu, Tong, Zhou, Pan, Liu, Kai, Yuan, Yali, Wang, Xiumin, Huang, Huawei, Wu, Dapeng Oliver
Format: Journal Article
Language:English
Published: New York IEEE 01.08.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Ad hoc networks
Algorithms
Buses (vehicles)
Computer science
Decision making
deep deterministic policy gradient algorithm
Deep learning
deep reinforcement learning
Heuristic algorithms
Internet of Things
Intersections
markov decision process
Multiagent systems
Pedestrians
Process control
Reinforcement learning
Traffic congestion
Traffic control
Traffic light control
Traffic signals
Training
Vehicle dynamics
Vehicles
vehicular network
ISSN:0018-9545, 1939-9359
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:As urban traffic condition is diverse and complicated, applying reinforcement learning to reduce traffic congestion becomes one of the hot and promising topics. Especially, how to coordinate the traffic light controllers of multiple intersections is a key challenge for multi-agent reinforcement learning (MARL). Most existing MARL studies are based on traditional <inline-formula><tex-math notation="LaTeX">Q</tex-math></inline-formula>-learning, but unstable environment leads to poor learning in the complicated and dynamic traffic scenarios. In this paper, we propose a novel multi-agent recurrent deep deterministic policy gradient (MARDDPG) algorithm based on deep deterministic policy gradient (DDPG) algorithm for traffic light control (TLC) in vehiclar networks. Specifically, the centralized learning in each critic network enables each agent to estimate the policies of other agents in the decision-making process and each agent can coordinate with each other, alleviating the problem of poor learning performance caused by environmental instability. The decentralized execution enables each agent to make decisions independently. We share parameters in actor networks to speed up the training process and reduce the memory footprint. The addition of LSTM is beneficial to alleviate the instability of the environment caused by partial observable state. We utilize surveillance cameras and vehicular networks to collect status information for each intersection. Unlike previous work, we have not only considered the vehicle but also considered the pedestrians waiting to pass through the intersection. Moreover, we also set different priorities for buses and ordinary vehicles. The experimental results in a vehicular network show that our method can run stably in various scenarios and coordinate multiple intersections, which significantly reduces vehicle congestion and pedestrian congestion.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9545
1939-9359
DOI:10.1109/TVT.2020.2997896