FedLight: Federated Reinforcement Learning for Autonomous Multi-Intersection Traffic Signal Control
Although Reinforcement Learning (RL) has been successfully applied in traffic control, it suffers from the problems of high average vehicle travel time and slow convergence to optimized solutions. This is because, due to the scalability restriction, most existing RL-based methods focus on the optimi...
Gespeichert in:
| Veröffentlicht in: | 2021 58th ACM/IEEE Design Automation Conference (DAC) S. 847 - 852 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Tagungsbericht |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
05.12.2021
|
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Although Reinforcement Learning (RL) has been successfully applied in traffic control, it suffers from the problems of high average vehicle travel time and slow convergence to optimized solutions. This is because, due to the scalability restriction, most existing RL-based methods focus on the optimization of individual intersections while the impact of their cooperation is neglected. Without taking all the correlated intersections as a whole into account, it is difficult to achieve global optimization goals for complex traffic scenarios. To address this issue, this paper proposes a novel federated reinforcement learning approach named FedLight to enable optimal signal control policy generation for multi-intersection traffic scenarios. Inspired by federated learning, our approach supports knowledge sharing among RL agents, whose models are trained using decentralized traffic data at intersections. Based on such model-level collaborations, both the overall convergence rate and control quality can be significantly improved. Comprehensive experimental results demonstrate that compared with the state-of-the-art techniques, our approach can not only achieve better average vehicle travel time for various multi-intersection configurations, but also converge to optimal solutions much faster. |
|---|---|
| AbstractList | Although Reinforcement Learning (RL) has been successfully applied in traffic control, it suffers from the problems of high average vehicle travel time and slow convergence to optimized solutions. This is because, due to the scalability restriction, most existing RL-based methods focus on the optimization of individual intersections while the impact of their cooperation is neglected. Without taking all the correlated intersections as a whole into account, it is difficult to achieve global optimization goals for complex traffic scenarios. To address this issue, this paper proposes a novel federated reinforcement learning approach named FedLight to enable optimal signal control policy generation for multi-intersection traffic scenarios. Inspired by federated learning, our approach supports knowledge sharing among RL agents, whose models are trained using decentralized traffic data at intersections. Based on such model-level collaborations, both the overall convergence rate and control quality can be significantly improved. Comprehensive experimental results demonstrate that compared with the state-of-the-art techniques, our approach can not only achieve better average vehicle travel time for various multi-intersection configurations, but also converge to optimal solutions much faster. |
| Author | Wei, Tongquan Ye, Yutong Chen, Mingsong Zhao, Wupan Hu, Shiyan |
| Author_xml | – sequence: 1 givenname: Yutong surname: Ye fullname: Ye, Yutong organization: East China Normal University,MoE Research Center of HW/SW Co-design,Shanghai,China,200062 – sequence: 2 givenname: Wupan surname: Zhao fullname: Zhao, Wupan organization: East China Normal University,MoE Research Center of HW/SW Co-design,Shanghai,China,200062 – sequence: 3 givenname: Tongquan surname: Wei fullname: Wei, Tongquan organization: East China Normal University,MoE Research Center of HW/SW Co-design,Shanghai,China,200062 – sequence: 4 givenname: Shiyan surname: Hu fullname: Hu, Shiyan organization: University of Southampton,School of Electronics and Computer Science,Southampton,United Kingdom,SO17 1BJ – sequence: 5 givenname: Mingsong surname: Chen fullname: Chen, Mingsong email: mschen@sei.ecnu.edu.cn organization: East China Normal University,MoE Research Center of HW/SW Co-design,Shanghai,China,200062 |
| BookMark | eNotkM1KAzEcxCMoqLVPIEJeYGu-k_VW1q_CiqD1XLLJPzWwTSSbHnx7F-xlZvgxzGGu0XnKCRC6o2RFKWnvH9cdNUSLFSOMrlppFNXyDC1bbahSUnCmBblEy2mKA1FEGjHrFXLP4Pu4_64PeE5QbAWPPyCmkIuDA6SKe7AlxbTHM8LrY80pH_Jxwm_HscZmkyqUCVyNOeFtsSFEhz_jPtkRdznVkscbdBHsOMHy5Av09fy07V6b_v1l0637xnLJa8OZMsS22jLuBBPeOqFsGCyA0kZ431LugEnvuRk4JS5YHaSca9QG4YfAF-j2fzcCwO6nxIMtv7vTFfwPcQtZSQ |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/DAC18074.2021.9586175 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9781665432740 1665432748 |
| EndPage | 852 |
| ExternalDocumentID | 9586175 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Key Research and Development Program of China funderid: 10.13039/501100012166 |
| GroupedDBID | 6IE 6IH ACM ALMA_UNASSIGNED_HOLDINGS CBEJK RIE RIO |
| ID | FETCH-LOGICAL-a353t-32680a97a23c424dac46afbaee6784dd913ce25dd38b310cfa7f55dac1af4dbf3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 24 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000766079700142&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:28:30 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a353t-32680a97a23c424dac46afbaee6784dd913ce25dd38b310cfa7f55dac1af4dbf3 |
| PageCount | 6 |
| ParticipantIDs | ieee_primary_9586175 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-12-05 |
| PublicationDateYYYYMMDD | 2021-12-05 |
| PublicationDate_xml | – month: 12 year: 2021 text: 2021-12-05 day: 05 |
| PublicationDecade | 2020 |
| PublicationTitle | 2021 58th ACM/IEEE Design Automation Conference (DAC) |
| PublicationTitleAbbrev | DAC |
| PublicationYear | 2021 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssib060584060 |
| Score | 2.3063695 |
| Snippet | Although Reinforcement Learning (RL) has been successfully applied in traffic control, it suffers from the problems of high average vehicle travel time and... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 847 |
| SubjectTerms | Advantage Actor-Critic Collaboration Collaborative work Design automation Federated Reinforcement Learning Neural Network Process control Reinforcement learning Scalability Traffic control Traffic Signal Control |
| Title | FedLight: Federated Reinforcement Learning for Autonomous Multi-Intersection Traffic Signal Control |
| URI | https://ieeexplore.ieee.org/document/9586175 |
| WOSCitedRecordID | wos000766079700142&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5t8eBJpRXf5ODRtM0m-_JWqosHKcUH9FbymJRettLu-vudZNeK4MXbEjYJZALfzGS-bwi55Y4Lo9OI6cgBBig5Z5lUCdNKIWABCJU3zSbS2SxbLPJ5h9ztuTAAEIrPYOg_w1u-3Zjap8pGeZwh4MZd0k3TpOFqfd8d_7qH2DRuSTp8nI8eJlPupV4wCIz4sJ37q4lKwJDi6H-7H5PBDxmPzvcwc0I6UPaJKcA--7j6nhZeDgI9RktfIMigmpDxo61y6oriEJ3UlWcvYJhPA-WWhUzgLtRhlRQByytJ0Nf1Cu8VnTbl6wPyXjy-TZ9Y2y-BKRGLiqEnlo1VnqpIGBlJq4xMlNMKABFJWpujWSCKrRWZRq_OOJW6OMbfuHLSaidOSa_clHBGqOI6MRbRPdG4FGAYa1RmIyFTIdC6_Jz0_QEtPxpJjGV7Nhd_D1-SQ2-DUAUSX5Feta3hmhyYz2q9294EO34BBZ2hOQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA61CnpSacW3OXh0281rH95KdalYS9EKvZW8tvSylXbr73eSXSuCF29L2CSQCXwzk_m-QeiW5IRpFdNA0dxCgJKSIOEyCpSUAFjWMplWzSbi0SiZTtNxA91tuTDWWl98Zjvu07_lm6XeuFRZNxUJAK7YQbuCcxpWbK3v2-Pe9wCdwpqmQ8K0-9DrEyf2AmEgJZ169q82Kh5FssP_7X-E2j90PDzeAs0xatiihXRmzdBF1vc4c4IQ4DMa_Gq9EKr2OT9ca6fOMQzh3qZ0_AUI9LEn3QY-F7j2lVgFBshyWhL4bTGHm4X7VQF7G71nj5P-IKg7JgSSCVYG4IsloUxjSZnmlBupeSRzJa0FTOLGpGAYS4UxLFHg1-lcxrkQ8BuROTcqZyeoWSwLe4qwJCrSBvA9UrCUhUBWy8RQxmPGwL7kDLXcAc0-KlGMWX02538P36D9weRlOBs-jZ4v0IGzh68JEZeoWa429grt6c9ysV5de5t-AZT5pIA |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2021+58th+ACM%2FIEEE+Design+Automation+Conference+%28DAC%29&rft.atitle=FedLight%3A+Federated+Reinforcement+Learning+for+Autonomous+Multi-Intersection+Traffic+Signal+Control&rft.au=Ye%2C+Yutong&rft.au=Zhao%2C+Wupan&rft.au=Wei%2C+Tongquan&rft.au=Hu%2C+Shiyan&rft.date=2021-12-05&rft.pub=IEEE&rft.spage=847&rft.epage=852&rft_id=info:doi/10.1109%2FDAC18074.2021.9586175&rft.externalDocID=9586175 |