Collaborative Task Offloading Optimization for Satellite Mobile Edge Computing Using Multi-Agent Deep Reinforcement Learning
Satellite mobile edge computing (SMEC) achieves efficient processing for space missions by deploying computing servers on low Earth orbit (LEO) satellites, which supplements a strong computing service for future satellite-terrestrial integrated networks. However, considering the spatio-temporal cons...
Uložené v:
| Vydané v: | IEEE transactions on vehicular technology Ročník 73; číslo 10; s. 15483 - 15498 |
|---|---|
| Hlavní autori: | , , , , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
New York
IEEE
01.10.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Predmet: | |
| ISSN: | 0018-9545, 1939-9359 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Satellite mobile edge computing (SMEC) achieves efficient processing for space missions by deploying computing servers on low Earth orbit (LEO) satellites, which supplements a strong computing service for future satellite-terrestrial integrated networks. However, considering the spatio-temporal constraints on large-scale LEO networks, inter-satellite cooperative computing is still challenging. In this paper, a multi-agent collaborative task offloading scheme for distributed SMEC is proposed. Facing the time-varying available satellites and service requirements, each autonomous satellite agent dynamically adjusts offloading decisions and resource allocations based on local observations. Furthermore, for evaluating the behavioral contribution of an agent to task completion, we adopt a deep reinforcement learning algorithm based on counterfactual multi-agent policy gradients (COMA) to optimize the strategy, which enables energy-efficient decisions satisfying the time and resource restrictions of SMEC. An actor-critic (AC) framework is effectively exploited to separately implement centralized training and distributed execution (CTDE) of the algorithm. We also redesign the actor structure by introducing an attention-based bidirectional long short-term memory network (Atten-BiLSTM) to explore the temporal characteristics of LEO networks. The simulation results show that the proposed scheme can effectively enable satellite autonomous collaborative computing in the distributed SMEC environment, and outperforms the benchmark algorithms. |
|---|---|
| AbstractList | Satellite mobile edge computing (SMEC) achieves efficient processing for space missions by deploying computing servers on low Earth orbit (LEO) satellites, which supplements a strong computing service for future satellite-terrestrial integrated networks. However, considering the spatio-temporal constraints on large-scale LEO networks, inter-satellite cooperative computing is still challenging. In this paper, a multi-agent collaborative task offloading scheme for distributed SMEC is proposed. Facing the time-varying available satellites and service requirements, each autonomous satellite agent dynamically adjusts offloading decisions and resource allocations based on local observations. Furthermore, for evaluating the behavioral contribution of an agent to task completion, we adopt a deep reinforcement learning algorithm based on counterfactual multi-agent policy gradients (COMA) to optimize the strategy, which enables energy-efficient decisions satisfying the time and resource restrictions of SMEC. An actor-critic (AC) framework is effectively exploited to separately implement centralized training and distributed execution (CTDE) of the algorithm. We also redesign the actor structure by introducing an attention-based bidirectional long short-term memory network (Atten-BiLSTM) to explore the temporal characteristics of LEO networks. The simulation results show that the proposed scheme can effectively enable satellite autonomous collaborative computing in the distributed SMEC environment, and outperforms the benchmark algorithms. |
| Author | Zhao, Hongbo Zhang, Hangyu Kaushik, Aryan Xu, Shenzhan Gao, Xiangqiang Liu, Rongke |
| Author_xml | – sequence: 1 givenname: Hangyu orcidid: 0000-0001-5544-0124 surname: Zhang fullname: Zhang, Hangyu email: zhanghangyu@buaa.edu.cn organization: School of Electronic and Information Engineering, Beihang University, Beijing, China – sequence: 2 givenname: Hongbo orcidid: 0000-0002-1196-4089 surname: Zhao fullname: Zhao, Hongbo email: bhzhb@buaa.edu.cn organization: School of Electronic and Information Engineering, Beihang University, Beijing, China – sequence: 3 givenname: Rongke orcidid: 0000-0003-3098-8649 surname: Liu fullname: Liu, Rongke email: rongke_liu@buaa.edu.cn organization: Shenzhen Institute, Beihang University, Shenzhen, China – sequence: 4 givenname: Aryan orcidid: 0000-0001-6252-4641 surname: Kaushik fullname: Kaushik, Aryan email: aryan.kaushik@sussex.ac.uk organization: School of Engineering and Informatics, University of Sussex, Brighton, U.K – sequence: 5 givenname: Xiangqiang orcidid: 0000-0002-2289-6229 surname: Gao fullname: Gao, Xiangqiang email: xggao@buaa.edu.cn organization: China Academy of Space Technology (Xi'an), Xi'an, China – sequence: 6 givenname: Shenzhan surname: Xu fullname: Xu, Shenzhan email: splayxu@buaa.edu.cn organization: School of Electronic and Information Engineering, Beihang University, Beijing, China |
| BookMark | eNp9kM9r2zAYhsVIYUnbew89CHZ2ql-OrWPIsraQEljTXs0n-XNQq1ierAw2-sfXXnoYO-wioU_v873wzMikDS0ScsXZnHOmb3bPu7lgQs2lYvlCiU9kyrXUmZa5npApY7zMdK7yz2TW9y_DUynNp-RtFbwHEyIk9xPpDvpXum0aH6B27Z5uu-QO7vfwGVrahEgfIaH3LiF9CMZ5pOt6j3QVDt0xjcBTP54PR59cttxjm-hXxI5-R9cOuMXDONogxHbIXZCzBnyPlx_3OXn6tt6t7rLN9vZ-tdxkVmiRstIKLAul80aXZYELgyCKWqnGWAmFERagLkHWtWFgJACrLUMlBVpjapujPCdfTnu7GH4csU_VSzjGdqisJOcFl7zMyyHFTikbQ99HbKouugPEXxVn1ei4GhxXo-Pqw_GALP5BrEt_ZKUIzv8PvD6BDhH_6sn1gikp3wEO_47S |
| CODEN | ITVTAB |
| CitedBy_id | crossref_primary_10_1007_s11227_024_06744_z crossref_primary_10_1038_s41598_025_93731_w crossref_primary_10_1016_j_eswa_2025_129702 crossref_primary_10_1109_TVT_2025_3526213 crossref_primary_10_3390_machines13060503 crossref_primary_10_1007_s10462_025_11340_5 crossref_primary_10_1109_JIOT_2025_3559984 crossref_primary_10_1007_s44443_025_00160_w crossref_primary_10_1109_JIOT_2025_3575158 crossref_primary_10_1109_JIOT_2025_3578620 crossref_primary_10_1109_TMC_2025_3539945 crossref_primary_10_1109_LWC_2025_3542085 crossref_primary_10_1016_j_adhoc_2025_103987 crossref_primary_10_3390_electronics14051016 crossref_primary_10_3390_app15084127 crossref_primary_10_1109_JIOT_2025_3584469 crossref_primary_10_3390_math13071183 crossref_primary_10_1016_j_iot_2025_101603 |
| Cites_doi | 10.1109/LCOMM.2015.2509993 10.1109/TSC.2022.3190562 10.1109/6GNet54646.2022.9830496 10.1007/s12652-020-02761-x 10.1109/MNET.105.2100614 10.1109/TGCN.2022.3186792 10.1109/MCOM.2019.1800155 10.1109/TNSM.2022.3141165 10.1137/s0363012901385691 10.1109/TWC.2023.3235997 10.1109/TMC.2022.3141080 10.21437/Interspeech.2014-80 10.1109/JIOT.2022.3233383 10.1109/TWC.2021.3100247 10.1109/TCOMM.2023.3296584 10.1109/JIOT.2021.3056569 10.1109/JIOT.2021.3063509 10.1155/2022/4193365 10.1109/MNET.011.1900369 10.1109/LCOMM.2021.3095227 10.1109/ISPDS51347.2020.00019 10.1609/aaai.v32i1.11794 10.1109/TNSE.2022.3159796 10.1109/JIOT.2021.3085129 10.1109/COMST.2022.3160697 10.1109/MWC.2019.1800299 10.1109/TWC.2021.3080578 10.3390/s20226442 10.1002/sys.21428 10.1145/3452296.3472932 10.1109/TAES.2021.3090914 10.1109/MNET.2018.1800052 10.1109/MCOM.2015.7060481 10.1109/ICCC55456.2022.9880821 10.1109/TVT.2022.3224765 10.1109/tnse.2022.3141728 10.1109/GLOBECOM38437.2019.9013467 10.3390/aerospace9050234 10.1109/TWC.2021.3050335 10.1109/MSP.2017.2743240 10.1109/TWC.2017.2647805 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024 |
| DBID | 97E RIA RIE AAYXX CITATION 7SP 8FD FR3 KR7 L7M |
| DOI | 10.1109/TVT.2024.3405642 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Electronics & Communications Abstracts Technology Research Database Engineering Research Database Civil Engineering Abstracts Advanced Technologies Database with Aerospace |
| DatabaseTitle | CrossRef Civil Engineering Abstracts Engineering Research Database Technology Research Database Advanced Technologies Database with Aerospace Electronics & Communications Abstracts |
| DatabaseTitleList | Civil Engineering Abstracts |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1939-9359 |
| EndPage | 15498 |
| ExternalDocumentID | 10_1109_TVT_2024_3405642 10596043 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Shenzhen Fundamental Research Project grantid: JCYJ20220818103413029 – fundername: Shenzhen Science and Technology R&D Funds grantid: JSGG20220831100602005 – fundername: Beijing Natural Science Foundation grantid: L202003 funderid: 10.13039/501100004826 |
| GroupedDBID | -~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 97E AAIKC AAJGR AAMNW AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 IAAWW IBMZZ ICLAB IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS RXW TAE TN5 VH1 AAYXX CITATION 7SP 8FD FR3 KR7 L7M |
| ID | FETCH-LOGICAL-c292t-8c2e87495f9887e6bea27d44fbc3a7b2caad8a3ddb0ab3aa0dc0e432ecbbdc5e3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 27 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001336949600106&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0018-9545 |
| IngestDate | Mon Jun 30 10:16:36 EDT 2025 Sat Nov 29 02:59:16 EST 2025 Tue Nov 18 22:27:42 EST 2025 Wed Aug 27 01:57:03 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 10 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c292t-8c2e87495f9887e6bea27d44fbc3a7b2caad8a3ddb0ab3aa0dc0e432ecbbdc5e3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-3098-8649 0000-0002-2289-6229 0000-0001-6252-4641 0000-0002-1196-4089 0000-0001-5544-0124 |
| PQID | 3117131858 |
| PQPubID | 85454 |
| PageCount | 16 |
| ParticipantIDs | proquest_journals_3117131858 crossref_primary_10_1109_TVT_2024_3405642 crossref_citationtrail_10_1109_TVT_2024_3405642 ieee_primary_10596043 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-10-01 |
| PublicationDateYYYYMMDD | 2024-10-01 |
| PublicationDate_xml | – month: 10 year: 2024 text: 2024-10-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on vehicular technology |
| PublicationTitleAbbrev | TVT |
| PublicationYear | 2024 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref35 ref34 ref15 ref37 ref14 ref36 ref31 ref30 ref11 ref33 ref10 ref32 ref2 ref1 ref17 ref39 ref16 ref38 ref19 ref18 ref24 ref23 ref26 ref25 ref20 Sutton (ref41) 1999 ref42 Yoon (ref12) 2017 ref22 ref21 ref43 ref28 ref27 ref29 ref8 ref7 ref9 ref4 ref3 ref6 ref5 ref40 |
| References_xml | – year: 2017 ident: ref12 article-title: Pointing system performance analysis for optical inter-satellite communication on cubesats – ident: ref17 doi: 10.1109/LCOMM.2015.2509993 – ident: ref28 doi: 10.1109/TSC.2022.3190562 – ident: ref37 doi: 10.1109/6GNet54646.2022.9830496 – ident: ref39 doi: 10.1007/s12652-020-02761-x – ident: ref21 doi: 10.1109/MNET.105.2100614 – ident: ref29 doi: 10.1109/TGCN.2022.3186792 – ident: ref32 doi: 10.1109/MCOM.2019.1800155 – ident: ref6 doi: 10.1109/TNSM.2022.3141165 – ident: ref42 doi: 10.1137/s0363012901385691 – ident: ref26 doi: 10.1109/TWC.2023.3235997 – ident: ref27 doi: 10.1109/TMC.2022.3141080 – ident: ref40 doi: 10.21437/Interspeech.2014-80 – start-page: 1057 volume-title: Proc. 12th Int. Conf. Neural Inf. Process. Syst. year: 1999 ident: ref41 article-title: Policy gradient methods for reinforcement learning with function approximation – ident: ref16 doi: 10.1109/JIOT.2022.3233383 – ident: ref20 doi: 10.1109/TWC.2021.3100247 – ident: ref22 doi: 10.1109/TCOMM.2023.3296584 – ident: ref36 doi: 10.1109/JIOT.2021.3056569 – ident: ref24 doi: 10.1109/JIOT.2021.3063509 – ident: ref30 doi: 10.1155/2022/4193365 – ident: ref1 doi: 10.1109/MNET.011.1900369 – ident: ref34 doi: 10.1109/LCOMM.2021.3095227 – ident: ref10 doi: 10.1109/ISPDS51347.2020.00019 – ident: ref33 doi: 10.1609/aaai.v32i1.11794 – ident: ref7 doi: 10.1109/TNSE.2022.3159796 – ident: ref8 doi: 10.1109/JIOT.2021.3085129 – ident: ref38 doi: 10.1109/COMST.2022.3160697 – ident: ref35 doi: 10.1109/MWC.2019.1800299 – ident: ref9 doi: 10.1109/TWC.2021.3080578 – ident: ref3 doi: 10.3390/s20226442 – ident: ref19 doi: 10.1002/sys.21428 – ident: ref4 doi: 10.1145/3452296.3472932 – ident: ref11 doi: 10.1109/TAES.2021.3090914 – ident: ref18 doi: 10.1109/MNET.2018.1800052 – ident: ref2 doi: 10.1109/MCOM.2015.7060481 – ident: ref15 doi: 10.1109/ICCC55456.2022.9880821 – ident: ref23 doi: 10.1109/TVT.2022.3224765 – ident: ref5 doi: 10.1109/tnse.2022.3141728 – ident: ref14 doi: 10.1109/GLOBECOM38437.2019.9013467 – ident: ref31 doi: 10.3390/aerospace9050234 – ident: ref13 doi: 10.1109/TWC.2021.3050335 – ident: ref25 doi: 10.1109/MSP.2017.2743240 – ident: ref43 doi: 10.1109/TWC.2017.2647805 |
| SSID | ssj0014491 |
| Score | 2.5563464 |
| Snippet | Satellite mobile edge computing (SMEC) achieves efficient processing for space missions by deploying computing servers on low Earth orbit (LEO) satellites,... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 15483 |
| SubjectTerms | Algorithms Collaboration Computation offloading Computational modeling Cooperative systems Decisions Deep learning Deep reinforcement learning distributed cooperative computing Edge computing Low earth orbit satellites Low earth orbits Machine learning Mobile computing Multi-access edge computing multi-agent deep reinforcement learning Multi-agent systems Multiagent systems Networks Optimization Redesign Resource allocation Resource management Satellite broadcasting Satellite communication Satellite mobile edge computing Satellite observation Satellites Space missions |
| Title | Collaborative Task Offloading Optimization for Satellite Mobile Edge Computing Using Multi-Agent Deep Reinforcement Learning |
| URI | https://ieeexplore.ieee.org/document/10596043 https://www.proquest.com/docview/3117131858 |
| Volume | 73 |
| WOSCitedRecordID | wos001336949600106&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1939-9359 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014491 issn: 0018-9545 databaseCode: RIE dateStart: 19670101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELagYoCBN6K85IGFIdSxncQZEQ8x8BIUxBbZ5wtCQIto6cSPx3ZcHkIgsWXwRVE--_ydffcdIdsiE5pnqk5S0DaRus4SI0uToGBWKltmZTiHvDkpzs7U7W15EYvVQy0MIobkM9z1j-Eu3_bh1R-VdTwXyJkUk2SyKPKmWOvjykDK2B4vdSvY8YLxnSQrO92brosEudwVjp7kkn_bg0JTlR-eOGwvR3P__LB5Mht5JN1rgF8gE9hbJDNf1AWXyNv-J8YjpF09eKDndf3YD2nz9Nw5i6dYhUkddaVXOqhzDpGe9o1zFvTQ3iFt2j54g5BdQEPFbrLnK7LoAeIzvcQgvgrhnJFGvda7ZXJ9dNjdP05is4UEeMmHiQKOqnDhUl06v4O5Qc0LK2VtQOjCcNDaKi2sNUwboTWzwFAKjmCMhQzFCmn1-j1cJbSAQtTWcQkwWqbMKgi696AgZxottEln_PsriErkviHGYxUiElZWDrDKA1ZFwNpk58PiuVHh-GPssgfoy7gGmzbZGENcxXU6qESauijdcRa19ovZOpn2b2_y9zZIa_jyiptkCkbD-8HLVpiC78le3GY |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT9wwEB61tFLpAfoAsTxaH3rpIaxjOxvniHgIxLJUbYq4RfZ4gqrCLmIXTv3x2I4XqKpW6i0HjxLls8ff2DPfAHyShTSi0G2Wo3GZMm2RWVXZjCR3SruqqOI55NmwHI30-Xn1JRWrx1oYIorJZ7QdHuNdvpvgbTgq6wcuMOBKPocXhVKCd-VaD5cGSqUGeblfw54ZzG8ledWvz2ofCwq1LT1BGSjx2y4U26r84YvjBnOw_J-f9gaWEpNkOx30b-EZjd_B6yf6gu_h1-4jynfEajP9yU7b9nISE-fZqXcXV6kOk3nyyr6ZqM85I3Yysd5dsH13Qaxr_BAMYn4BizW72U6oyWJ7RNfsK0X5VYwnjSwptl6swPeD_Xr3MEvtFjIUlZhlGgXp0gdMbeU9Dw0sGVE6pVqL0pRWoDFOG-mc5cZKY7hDTkoKQmsdFiRXYWE8GdMasBJL2TrPJtAalXOnMSrfo8YBN-SwB_35728waZGHlhiXTYxJeNV4wJoAWJMA68HnB4vrTofjH2NXAkBPxnXY9GBzDnGTVuq0kXnu43TPWvT6X8w-wqvD-mTYDI9GxxuwGN7UZfNtwsLs5pa24CXezX5Mbz7E6XgPYzHfrQ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Collaborative+Task+Offloading+Optimization+for+Satellite+Mobile+Edge+Computing+Using+Multi-Agent+Deep+Reinforcement+Learning&rft.jtitle=IEEE+transactions+on+vehicular+technology&rft.au=Zhang%2C+Hangyu&rft.au=Zhao%2C+Hongbo&rft.au=Liu%2C+Rongke&rft.au=Kaushik%2C+Aryan&rft.date=2024-10-01&rft.pub=IEEE&rft.issn=0018-9545&rft.volume=73&rft.issue=10&rft.spage=15483&rft.epage=15498&rft_id=info:doi/10.1109%2FTVT.2024.3405642&rft.externalDocID=10596043 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9545&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9545&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9545&client=summon |