An inductive heterogeneous graph attention-based multi-agent deep graph infomax algorithm for adaptive traffic signal control

•An algorithm based on heterogeneous GNNs is proposed for feature fusion.•A mutual-information loss function is proposed for training.•Based on above methods, an IHA-MDGI algorithm is proposed for traffic signal control.•Experiments show the IHA-MDGI algorithm outperforms the state-of-the-art algori...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	Information fusion Ročník 88; s. 249 - 262
Hlavní autoři:	Yang, Shantian, Yang, Bo
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	Elsevier B.V 01.12.2022
Témata:	Adaptive traffic signal control Attention mechanism Heterogeneous feature fusion Multi-agent reinforcement learning Mutual information Multi-agent reinforcement learning Adaptive traffic signal control Attention mechanism Heterogeneous feature fusion Mutual information
ISSN:	1566-2535, 1872-6305
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Popis
Shrnutí:	•An algorithm based on heterogeneous GNNs is proposed for feature fusion.•A mutual-information loss function is proposed for training.•Based on above methods, an IHA-MDGI algorithm is proposed for traffic signal control.•Experiments show the IHA-MDGI algorithm outperforms the state-of-the-art algorithms. Adaptive traffic signal control (ATSC) facilitates alleviating traffic congestion. Multi-agent deep reinforcement learning (MDRL) is a new promising algorithm for ATSC, and Graph Neural Networks (GNNs) further promote its learning ability. However, there are some drawbacks in the state-of-the-art MDRL algorithms. (1) These algorithms cannot effectively fuse diverse heterogeneous information of traffic networks due to adopting homogeneous GNNs; (2) These algorithms cannot be effectively trained due to merely adopting MDRL loss functions. In this paper, we propose an Inductive Heterogeneous graph Attention-based Multi-agent Deep Graph Infomax (IHA-MDGI) algorithm for ATSC. The proposed IHA-MDGI algorithm conducts both feature fusion via a proposed Inductive Heterogeneous graph Attention (IHA) algorithm and training via a proposed Multi-agent Deep Graph Infomax (MDGI) framework. Specifically, (1) Unlike the algorithms which adopt homogeneous GNNs, in the IHA algorithm, heterogeneous GNNs are designed to fuse both heterogeneous structural information and heterogeneous features of traffic networks, which aims to acquire heterogeneous information embeddings of traffic networks. (2) In the MDGI framework, the acquired embeddings are used to calculate the signal-control policies and Q-value for each agent, and then a mutual-information loss function is designed, which combines with the MDRL loss function to jointly train the whole algorithm. The designed mutual-information loss function focuses on maximizing mutual information between input (i.e., heterogeneous information embeddings) and output (i.e., Q-value), which can produce cooperative signal-control policies and maximize Q-value. We conduct the experiments in both real-traffic and synthetic-traffic networks under the time-varying traffic flows, and the results demonstrate that IHA-MDGI algorithm outperforms the state-of-the-art MDRL algorithms about multiple metrics.
ISSN:	1566-2535 1872-6305
DOI:	10.1016/j.inffus.2022.08.001