Hierarchical Task Offloading for Vehicular Fog Computing Based on Multi-Agent Deep Reinforcement Learning
Vehicular fog computing (VFC) has been expected as a promising architecture that can make full use of computing resources of idle vehicles to increase computing capability. However, most current VFC architectures only focus on the local region and ignore the spatio-temporal heterogeneity of computin...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on wireless communications Jg. 23; H. 4; S. 3074 - 3085 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
New York
IEEE
01.04.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Schlagworte: | |
| ISSN: | 1536-1276, 1558-2248 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Vehicular fog computing (VFC) has been expected as a promising architecture that can make full use of computing resources of idle vehicles to increase computing capability. However, most current VFC architectures only focus on the local region and ignore the spatio-temporal heterogeneity of computing resources, resulting in that some regions have idle computing resources while others cannot satisfy the requirements of tasks. To further improve the overall computing resource utilization in the whole network, in this work, we propose a hierarchical VFC architecture, where neighboring regions can share their idle computing resources. Considering the high complexity of both inter- and intra-region cooperative task offloading in such a hierarchical VFC architecture, we put forward a distributed task offloading strategy based on multi-agent reinforcement learning in which the multi-agent reinforcement learning method is designed to learn each task vehicle's offloading strategy in a distributed manner. Moreover, to tackle the inefficiency caused by the multi-agent credit assignment problem, we provide the counterfactual multi-agent reinforcement learning approach which exploits a counterfactual baseline to evaluate the action of each agent. Simulation results validate that the proposed hierarchical VFC architecture can effectively improve the global task computing efficiency and the proposed mechanism outperforms the baseline algorithms. |
|---|---|
| AbstractList | Vehicular fog computing (VFC) has been expected as a promising architecture that can make full use of computing resources of idle vehicles to increase computing capability. However, most current VFC architectures only focus on the local region and ignore the spatio-temporal heterogeneity of computing resources, resulting in that some regions have idle computing resources while others cannot satisfy the requirements of tasks. To further improve the overall computing resource utilization in the whole network, in this work, we propose a hierarchical VFC architecture, where neighboring regions can share their idle computing resources. Considering the high complexity of both inter- and intra-region cooperative task offloading in such a hierarchical VFC architecture, we put forward a distributed task offloading strategy based on multi-agent reinforcement learning in which the multi-agent reinforcement learning method is designed to learn each task vehicle’s offloading strategy in a distributed manner. Moreover, to tackle the inefficiency caused by the multi-agent credit assignment problem, we provide the counterfactual multi-agent reinforcement learning approach which exploits a counterfactual baseline to evaluate the action of each agent. Simulation results validate that the proposed hierarchical VFC architecture can effectively improve the global task computing efficiency and the proposed mechanism outperforms the baseline algorithms. |
| Author | Wei, Zhiwei Hou, Yukai Cheng, Xiang Zhang, Rongqing Yang, Liuqing |
| Author_xml | – sequence: 1 givenname: Yukai orcidid: 0000-0003-4640-1582 surname: Hou fullname: Hou, Yukai email: 2131508@tongji.edu.cn organization: School of Software Engineering, Tongji University, Shanghai, China – sequence: 2 givenname: Zhiwei orcidid: 0000-0001-9593-0013 surname: Wei fullname: Wei, Zhiwei email: 2031563@tongji.edu.cn organization: School of Software Engineering, Tongji University, Shanghai, China – sequence: 3 givenname: Rongqing orcidid: 0000-0003-3774-6247 surname: Zhang fullname: Zhang, Rongqing email: rongqingz@tongji.edu.cn organization: School of Software Engineering, Tongji University, Shanghai, China – sequence: 4 givenname: Xiang orcidid: 0000-0002-5943-0326 surname: Cheng fullname: Cheng, Xiang email: xiangcheng@pku.edu.cn organization: School of Electronics, Peking University, Beijing, China – sequence: 5 givenname: Liuqing orcidid: 0000-0003-0231-6837 surname: Yang fullname: Yang, Liuqing email: lqyang@ust.hk organization: Internet of Things Thrust and Intelligent Transportation Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China |
| BookMark | eNp9kDFPwzAQhS1UJNrCzsBgiTnFPsdJOpZCKVJRJVRgjBznXFxSuzjJwL8nURkQA9Od7r3vnvRGZOC8Q0IuOZtwzqY3m7f5BBiIiRBMCuAnZMilzCKAOBv0u0giDmlyRkZ1vWOMp4mUQ2KXFoMK-t1qVdGNqj_o2pjKq9K6LTU-0FfstLZSgS78ls79_tA2vXaraiypd_SprRobzbboGnqHeKDPaF1Hatz3pxWq4DrgnJwaVdV48TPH5GVxv5kvo9X64XE-W0UaptBEscYsLZnkwE2pFcYqK3SBRsQFT9CUosRYghZpqkWhDMRTVInkqSlMCkaUYkyuj38PwX-2WDf5zrfBdZG5YEJCBpBlnYsdXTr4ug5o8kOwexW-cs7yvtC8KzTvC81_Cu2Q5A-ibaMa610TlK3-A6-OoEXEXzkAMpmm4huHX4ZC |
| CODEN | ITWCAX |
| CitedBy_id | crossref_primary_10_1109_JIOT_2025_3545747 crossref_primary_10_1007_s11227_023_05698_y crossref_primary_10_1049_itr2_70083 crossref_primary_10_1109_TMC_2025_3546263 crossref_primary_10_1109_OJCOMS_2024_3457023 crossref_primary_10_1016_j_future_2025_107950 crossref_primary_10_1049_cmu2_70064 crossref_primary_10_64362_zjse_7 crossref_primary_10_1016_j_adhoc_2025_103936 crossref_primary_10_1016_j_vehcom_2024_100862 crossref_primary_10_1109_JIOT_2024_3476476 crossref_primary_10_1016_j_adhoc_2025_104006 crossref_primary_10_1016_j_vehcom_2023_100626 crossref_primary_10_1007_s11277_024_11373_z crossref_primary_10_1109_JIOT_2025_3526150 crossref_primary_10_3390_fi16080278 |
| Cites_doi | 10.1109/TVT.2020.3041929 10.1109/ACCESS.2019.2893571 10.1016/j.future.2019.01.059 10.1109/VTC2022-Spring54318.2022.9860775 10.1109/MNET.001.1900527 10.1109/ACCESS.2020.3029169 10.1109/tmc.2022.3211882 10.1109/ACCESS.2019.2900530 10.1109/TVT.2017.2714704 10.1109/JIOT.2021.3104324 10.1109/ACCESS.2018.2872750 10.1109/JIOT.2019.2949602 10.1109/TVT.2020.2970763 10.1109/ACCESS.2019.2963051 10.1109/IWCMC.2019.8766579 10.1109/MWC.2019.1700441 10.1109/MNET.001.1900200 10.1109/TWC.2020.3007805 10.1109/JIOT.2022.3182163 10.1109/ISCC58397.2023.10217881 10.1109/TVT.2019.2894851 10.1109/TVT.2018.2868013 10.1109/JIOT.2019.2958400 10.1109/JIOT.2020.2975496 10.1109/JIOT.2019.2946426 10.1109/MCOM.2017.1700208 10.1109/CloudCom.2013.89 10.1109/MNET.001.1900512 10.1109/MNET.011.2000572 10.1109/TVT.2016.2532863 10.1109/TNET.2015.2487344 10.1109/TVT.2016.2591558 10.1109/WCNC49053.2021.9417450 10.1109/MNET.2018.1700442 10.1109/MWC.001.1900317 10.1109/TPDS.2020.3046737 10.1109/TVT.2013.2245156 10.1109/MNET.2018.1700105 10.1109/MNET.001.1800510 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TWC.2023.3305321 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1558-2248 |
| EndPage | 3085 |
| ExternalDocumentID | 10_1109_TWC_2023_3305321 10225697 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Key Research and Development Program of China grantid: 2021ZD0112700 funderid: 10.13039/501100012166 – fundername: Fundamental Research Funds for the Central Universities funderid: 10.13039/501100012226 – fundername: National Natural Science Foundation of China grantid: 62271351; 62125101; 62341101 funderid: 10.13039/501100001809 |
| GroupedDBID | -~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AIBXA AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 IES IFIPE IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c292t-4ce87d05121fdcae4a8bcbef34b16efd3de452c377c3baf249ea6517fbf72f3d3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 26 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001201360000012&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1536-1276 |
| IngestDate | Fri Jul 25 10:18:51 EDT 2025 Sat Nov 29 06:23:58 EST 2025 Tue Nov 18 22:32:10 EST 2025 Wed Aug 27 02:17:15 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 4 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c292t-4ce87d05121fdcae4a8bcbef34b16efd3de452c377c3baf249ea6517fbf72f3d3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-4640-1582 0000-0003-3774-6247 0000-0001-9593-0013 0000-0003-0231-6837 0000-0002-5943-0326 |
| PQID | 3035282288 |
| PQPubID | 105736 |
| PageCount | 12 |
| ParticipantIDs | crossref_primary_10_1109_TWC_2023_3305321 crossref_citationtrail_10_1109_TWC_2023_3305321 proquest_journals_3035282288 ieee_primary_10225697 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-April 2024-4-00 20240401 |
| PublicationDateYYYYMMDD | 2024-04-01 |
| PublicationDate_xml | – month: 04 year: 2024 text: 2024-April |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on wireless communications |
| PublicationTitleAbbrev | TWC |
| PublicationYear | 2024 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref35 ref12 ref34 ref15 ref37 ref14 ref36 ref31 ref30 ref11 ref33 ref10 ref32 ref2 ref1 ref17 ref39 ref16 ref38 ref19 ref18 ref24 ref23 ref26 ref25 ref20 ref22 ref21 ref28 ref27 ref29 ref8 ref7 ref9 ref4 ref3 ref6 ref5 |
| References_xml | – ident: ref2 doi: 10.1109/TVT.2020.3041929 – ident: ref35 doi: 10.1109/ACCESS.2019.2893571 – ident: ref30 doi: 10.1016/j.future.2019.01.059 – ident: ref29 doi: 10.1109/VTC2022-Spring54318.2022.9860775 – ident: ref21 doi: 10.1109/MNET.001.1900527 – ident: ref17 doi: 10.1109/ACCESS.2020.3029169 – ident: ref32 doi: 10.1109/tmc.2022.3211882 – ident: ref24 doi: 10.1109/ACCESS.2019.2900530 – ident: ref12 doi: 10.1109/TVT.2017.2714704 – ident: ref13 doi: 10.1109/JIOT.2021.3104324 – ident: ref7 doi: 10.1109/ACCESS.2018.2872750 – ident: ref15 doi: 10.1109/JIOT.2019.2949602 – ident: ref1 doi: 10.1109/TVT.2020.2970763 – ident: ref18 doi: 10.1109/ACCESS.2019.2963051 – ident: ref20 doi: 10.1109/IWCMC.2019.8766579 – ident: ref3 doi: 10.1109/MWC.2019.1700441 – ident: ref26 doi: 10.1109/MNET.001.1900200 – ident: ref28 doi: 10.1109/TWC.2020.3007805 – ident: ref11 doi: 10.1109/JIOT.2022.3182163 – ident: ref4 doi: 10.1109/ISCC58397.2023.10217881 – ident: ref6 doi: 10.1109/TVT.2019.2894851 – ident: ref9 doi: 10.1109/TVT.2018.2868013 – ident: ref16 doi: 10.1109/JIOT.2019.2958400 – ident: ref14 doi: 10.1109/JIOT.2020.2975496 – ident: ref27 doi: 10.1109/JIOT.2019.2946426 – ident: ref34 doi: 10.1109/MCOM.2017.1700208 – ident: ref37 doi: 10.1109/CloudCom.2013.89 – ident: ref25 doi: 10.1109/MNET.001.1900512 – ident: ref23 doi: 10.1109/MNET.011.2000572 – ident: ref31 doi: 10.1109/TVT.2016.2532863 – ident: ref36 doi: 10.1109/TNET.2015.2487344 – ident: ref39 doi: 10.1109/TVT.2016.2591558 – ident: ref19 doi: 10.1109/WCNC49053.2021.9417450 – ident: ref5 doi: 10.1109/MNET.2018.1700442 – ident: ref8 doi: 10.1109/MWC.001.1900317 – ident: ref38 doi: 10.1109/TPDS.2020.3046737 – ident: ref33 doi: 10.1109/TVT.2013.2245156 – ident: ref10 doi: 10.1109/MNET.2018.1700105 – ident: ref22 doi: 10.1109/MNET.001.1800510 |
| SSID | ssj0017655 |
| Score | 2.5505521 |
| Snippet | Vehicular fog computing (VFC) has been expected as a promising architecture that can make full use of computing resources of idle vehicles to increase... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 3074 |
| SubjectTerms | Algorithms Complexity theory Computation offloading Computer architecture counterfactual multi-agent Deep learning Edge computing Heterogeneity hierarchical task offloading Idling multi-agent reinforcement learning Multiagent systems Operations research Reinforcement learning Resource management Resource utilization Servers Task analysis Vehicular fog computing |
| Title | Hierarchical Task Offloading for Vehicular Fog Computing Based on Multi-Agent Deep Reinforcement Learning |
| URI | https://ieeexplore.ieee.org/document/10225697 https://www.proquest.com/docview/3035282288 |
| Volume | 23 |
| WOSCitedRecordID | wos001201360000012&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2248 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0017655 issn: 1536-1276 databaseCode: RIE dateStart: 20020101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELYAMcDAG1EoyAMLQ_qwnTgey6NiQIBQgW6RH2eIqJKqLfx-bCetQAgktkixoyifL-fz3X0fQqdpLCzxfNqECBYxABYp7exKWg7OPcoEIFDm3_Db23Q4FPd1s3rohQGAUHwGLX8Zcvmm1O_-qKzto5M4EXwZLXOeVM1ai5QBT4LEqbNgLyzDFznJjmgPni9aXia85YL3mJLuNx8URFV-_ImDe-lv_vPFttBGvY_EvQr4bbQExQ5a_8IuuIvy69x3FwexkxEeyOkbvrN2VIaqeew2q_gJXvNQh4r75QuuBB78vXPn2gwuCxzac6Oeb7_ClwBj_ACBaVWHQ0Vck7O-7KHH_tXg4jqqlRUiTQSZRUxDyo2zR9K1RktgMlVagaVMdROwhhpgMdGUc02VtC5EA5nEXW6V5cRSQ_fRSlEWcIBwYkwsaRx491mspGKmY4WwjEhlLaQN1J5_60zXtONe_WKUhfCjIzKHTubRyWp0GuhsMWNcUW78MXbPo_FlXAVEAzXneGa1UU4z6rlf3YYoTQ9_mXaE1tzT68qcJlqZTd7hGK3qj1k-nZyE9fYJiG3T8w |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT9wwEB4VWgk49MFDbEupD71wyD78iOMjBVZbsd1WaFu4RX6MYcVqg9iF31_bCSsqBFJvkWIrUT5PxuOZ-T6Ar4VQnkY-bUoVzzgiz4wNdqW9xOAedY6YKPOHcjQqLi7Ur6ZZPfXCIGIqPsN2vEy5fFfZu3hU1onRiciVXIHXgnPardu1lkkDmSeR02DDUVpGLrOSXdUZnx-1o1B4O4TvgtHeP14oyao8-RcnB9N_95-v9h7eNjtJclhD_wFe4WwTNh7xC27BZDCJ_cVJ7mRKxnp-TX56P61S3TwJ21XyB68mqRKV9KtLUks8xHvfgnNzpJqR1KCbHcYGLHKMeEPOMHGt2nSsSBp61stt-N0_GR8NskZbIbNU0UXGLRbSBYukPe-sRq4LYw16xk0vR--YQy6oZVJaZrQPQRrqXPSkN15SzxzbgdVZNcNdILlzQjORmPe5MNpw1_VKeU618R6LFnQevnVpG-LxqH8xLVMA0lVlQKeM6JQNOi04WM64qUk3Xhi7HdF4NK4GogV7D3iWjVnOSxbZX8OWqCg-PjPtC6wNxj-G5fD76PQTrIcnNXU6e7C6uL3Dz_DG3i8m89v9tPb-ArWa1zo |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Hierarchical+Task+Offloading+for+Vehicular+Fog+Computing+Based+on+Multi-Agent+Deep+Reinforcement+Learning&rft.jtitle=IEEE+transactions+on+wireless+communications&rft.au=Hou%2C+Yukai&rft.au=Wei%2C+Zhiwei&rft.au=Zhang%2C+Rongqing&rft.au=Cheng%2C+Xiang&rft.date=2024-04-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=1536-1276&rft.eissn=1558-2248&rft.volume=23&rft.issue=4&rft.spage=3074&rft_id=info:doi/10.1109%2FTWC.2023.3305321&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1536-1276&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1536-1276&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1536-1276&client=summon |