An Online Distributed Satellite Cooperative Observation Scheduling Algorithm Based on Multiagent Deep Reinforcement Learning

The provision of real-time information services is one of the crucial functions of satellites. In comparison with the centralized scheduling, the distributed scheduling can provide better robustness and extendibility. However, the existing distributed satellite scheduling algorithms require a large...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE geoscience and remote sensing letters Ročník 18; číslo 11; s. 1901 - 1905
Hlavní autoři:	Dalin, Li, Haijiao, Wang, Zhen, Yang, Yanfeng, Gu, Shi, Shen
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	Piscataway IEEE 01.11.2021 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:	Aerospace electronics Algorithms Communication Decisions Deep learning Deep reinforcement learning (DRL) distributed scheduling Games imaging satellites Information services Machine learning Markov processes Multiagent systems neural network Policies Real time real-time scheduling Real-time systems Reinforcement Response time Satellite observation Satellites Scheduling Task analysis Task scheduling
ISSN:	1545-598X, 1558-0571
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Abstract	The provision of real-time information services is one of the crucial functions of satellites. In comparison with the centralized scheduling, the distributed scheduling can provide better robustness and extendibility. However, the existing distributed satellite scheduling algorithms require a large amount of communication between satellites to coordinate tasks, which makes it difficult to support scheduling in real-time. This letter proposes a multiagent deep reinforcement learning (MADRL)-based method to solve the problem of scheduling real-time multisatellite cooperative observation. The method enables satellites to share their decision policy, but it is not necessary to share data on the decisions they make or data on their current internal state. The satellites can use the decision policy to infer the decisions of other satellites to decide whether to accept a task when they receive a new request for observations. In this way, our method can significantly reduce the communication overhead and improve the response time. The pillar of the architecture is a multiagent deep deterministic policy gradient network. Our simulation results show that the proposed method is stable and effective. In comparison with the Contract Net Protocol method, our algorithm can reduce the communication overhead and achieve better use of satellite resources.
AbstractList	The provision of real-time information services is one of the crucial functions of satellites. In comparison with the centralized scheduling, the distributed scheduling can provide better robustness and extendibility. However, the existing distributed satellite scheduling algorithms require a large amount of communication between satellites to coordinate tasks, which makes it difficult to support scheduling in real-time. This letter proposes a multiagent deep reinforcement learning (MADRL)-based method to solve the problem of scheduling real-time multisatellite cooperative observation. The method enables satellites to share their decision policy, but it is not necessary to share data on the decisions they make or data on their current internal state. The satellites can use the decision policy to infer the decisions of other satellites to decide whether to accept a task when they receive a new request for observations. In this way, our method can significantly reduce the communication overhead and improve the response time. The pillar of the architecture is a multiagent deep deterministic policy gradient network. Our simulation results show that the proposed method is stable and effective. In comparison with the Contract Net Protocol method, our algorithm can reduce the communication overhead and achieve better use of satellite resources.
Author	Yanfeng, Gu Shi, Shen Zhen, Yang Dalin, Li Haijiao, Wang
Author_xml	– sequence: 1 givenname: Li orcidid: 0000-0003-4456-8645 surname: Dalin fullname: Dalin, Li email: lidalin@nssc.ac.cn organization: School of Electronics and information Engineering, Harbin Institute of Technology, Harbin, China – sequence: 2 givenname: Wang surname: Haijiao fullname: Haijiao, Wang email: youfang.whj@alibaba-inc.com organization: Key Laboratory of Electronics and Information Technology for Space Systems, National Space Science Center, Chinese Academy of Sciences, Beijing, China – sequence: 3 givenname: Yang surname: Zhen fullname: Zhen, Yang organization: Key Laboratory of Electronics and Information Technology for Space Systems, National Space Science Center, Chinese Academy of Sciences, Beijing, China – sequence: 4 givenname: Gu orcidid: 0000-0003-1625-7989 surname: Yanfeng fullname: Yanfeng, Gu email: guyf@hit.edu.cn organization: School of Electronics and information Engineering, Harbin Institute of Technology, Harbin, China – sequence: 5 givenname: Shen orcidid: 0000-0001-9126-229X surname: Shi fullname: Shi, Shen organization: Faculty of Geographical Science, Beijing Normal University, Beijing, China
BookMark	eNp9kN9LwzAQgINM0Kl_gPgS8LkzSZs2fZybv2Ay2BR8K2l7nRldMpN0IPjHm7rhgw8-3V3uvjvyDdFAGw0IXVIyopTkN7OHxXLECCOjmJBcsPgInVLORUR4Rgd9nvCI5-LtBA2dWxPCEiGyU_Q11niuW6UBT5XzVpWdhxovpYe2VR7wxJgtWOnVDvC8dGB3ITcaL6t3qLsArvC4XRmr_PsG30oX4NB97lqv5Aq0x1OALV6A0o2xFWz6pxlIqwN5jo4b2Tq4OMQz9Hp_9zJ5jGbzh6fJeBZVMc99RGnCmozXHIBVJFRZmjFeUlbzOmkaGvO4SbOS1w0LnTIRmYghASZFDTSpRXyGrvd7t9Z8dOB8sTad1eFkwbhIU0JJSsIU3U9V1jhnoSm2Vm2k_SwoKXrJRS-56CUXB8mByf4wlfI_gryVqv2XvNqTCgB-L-WUs_DB-BuTwY07
CODEN	IGRSBY
CitedBy_id	crossref_primary_10_3390_s23042225 crossref_primary_10_1016_j_eswa_2025_128502 crossref_primary_10_1080_17538947_2025_2554310 crossref_primary_10_1016_j_asr_2024_02_038 crossref_primary_10_3390_systems12110465 crossref_primary_10_1016_j_aei_2024_102915 crossref_primary_10_3390_s25061707 crossref_primary_10_1016_j_asr_2024_06_003 crossref_primary_10_1109_TAES_2022_3210073 crossref_primary_10_3390_a16020114 crossref_primary_10_1109_TAES_2024_3425389 crossref_primary_10_1016_j_asr_2025_07_047 crossref_primary_10_1016_j_asr_2024_07_044 crossref_primary_10_1007_s11432_022_3636_1 crossref_primary_10_1109_TAES_2022_3203664 crossref_primary_10_1109_JIOT_2024_3407123 crossref_primary_10_3390_e24040470 crossref_primary_10_1007_s42064_024_0253_1 crossref_primary_10_3390_s22197387 crossref_primary_10_3390_s24154938 crossref_primary_10_1587_transfun_2025EAL2009 crossref_primary_10_1109_JSAC_2023_3242700 crossref_primary_10_1587_transfun_2024EAL2106 crossref_primary_10_1109_ACCESS_2024_3462765 crossref_primary_10_1016_j_eswa_2025_128350 crossref_primary_10_1155_2022_1927937 crossref_primary_10_1016_j_aei_2024_102362 crossref_primary_10_3390_aerospace11080643 crossref_primary_10_3390_rs17121972 crossref_primary_10_1016_j_asr_2025_05_067 crossref_primary_10_1088_1742_6596_2746_1_012040
Cites_doi	10.1016/j.cja.2018.12.018 10.1007/978-3-642-26001-8_58 10.3390/s18051649 10.1109/LGRS.2017.2702137 10.2514/6.1999-4479 10.1016/B978-0-934613-63-7.50009-7 10.1016/j.eswa.2009.07.079 10.1016/j.actaastro.2018.11.001 10.1016/S0004-3702(02)00382-X 10.1007/BF02460000 10.2514/6.2018-2500 10.3390/s18041297 10.1016/B978-1-55860-307-3.50049-6 10.1109/ICNC.2015.7378139 10.3390/s17102347 10.1007/s10458-010-9145-2
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 7TG 7UA 8FD C1K F1W FR3 H8D H96 JQ2 KL. KR7 L.G L7M L~C L~D
DOI	10.1109/LGRS.2020.3009823
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Meteorological & Geoastrophysical Abstracts Water Resources Abstracts Technology Research Database Environmental Sciences and Pollution Management ASFA: Aquatic Sciences and Fisheries Abstracts Engineering Research Database Aerospace Database Aquatic Science & Fisheries Abstracts (ASFA) 2: Ocean Technology, Policy & Non-Living Resources ProQuest Computer Science Collection Meteorological & Geoastrophysical Abstracts - Academic Civil Engineering Abstracts Aquatic Science & Fisheries Abstracts (ASFA) Professional Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Civil Engineering Abstracts Aquatic Science & Fisheries Abstracts (ASFA) Professional Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Water Resources Abstracts Environmental Sciences and Pollution Management Computer and Information Systems Abstracts Professional Aerospace Database Meteorological & Geoastrophysical Abstracts Aquatic Science & Fisheries Abstracts (ASFA) 2: Ocean Technology, Policy & Non-Living Resources ASFA: Aquatic Sciences and Fisheries Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Meteorological & Geoastrophysical Abstracts - Academic
DatabaseTitleList	Civil Engineering Abstracts
Database_xml	– sequence: 1 dbid: RIE name: IEEE Xplore / Electronic Library Online (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Geography Geology
EISSN	1558-0571
EndPage	1905
ExternalDocumentID	10_1109_LGRS_2020_3009823 9152114
Genre	orig-research
GrantInformation_xml	– fundername: Key Programs of the Chinese Academy of Sciences grantid: ZDRW-KT-2016-2 funderid: 10.13039/501100002367
GroupedDBID	0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACIWK AENEX AETIX AFRAH AGQYO AGSQL AHBIQ AIBXA AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 EBS EJD HZ~ H~9 IFIPE IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS ~02 AAYXX CITATION 7SC 7SP 7TG 7UA 8FD C1K F1W FR3 H8D H96 JQ2 KL. KR7 L.G L7M L~C L~D
ID	FETCH-LOGICAL-c359t-1142f75d5ee2c011476725b12d5d4ff1353f67b5df2767b48783e4e2a8de14d83
IEDL.DBID	RIE
ISICitedReferencesCount	39
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000711828000012&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1545-598X
IngestDate	Mon Jun 30 13:43:09 EDT 2025 Tue Nov 18 22:30:42 EST 2025 Sat Nov 29 05:54:03 EST 2025 Wed Aug 27 02:28:40 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	11
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c359t-1142f75d5ee2c011476725b12d5d4ff1353f67b5df2767b48783e4e2a8de14d83
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0003-1625-7989 0000-0001-9126-229X 0000-0003-4456-8645
PQID	2586601060
PQPubID	75725
PageCount	5
ParticipantIDs	crossref_primary_10_1109_LGRS_2020_3009823 crossref_citationtrail_10_1109_LGRS_2020_3009823 ieee_primary_9152114 proquest_journals_2586601060
PublicationCentury	2000
PublicationDate	2021-11-01
PublicationDateYYYYMMDD	2021-11-01
PublicationDate_xml	– month: 11 year: 2021 text: 2021-11-01 day: 01
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE geoscience and remote sensing letters
PublicationTitleAbbrev	LGRS
PublicationYear	2021
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref15 ref14 lemaitre (ref7) 1998 ref10 lowe (ref30) 2017 ref2 schwartz (ref27) 2014 liu (ref12) 2014; 10 ref16 zhoupeng (ref17) 2013 colby (ref26) 2013 ref18 mozer (ref23) 2005; 1054 songyang (ref31) 2018 ref20 he (ref33) 2011 ref22 li (ref1) 2016; 20 ref21 matignon (ref25) 2014; 371 ref28 ref29 verfaillie (ref19) 2006 jiang (ref8) 2013; 35 ref9 peng (ref24) 2017 ref4 wen (ref6) 2001; 4 ref3 ref5 goldsborough (ref32) 2016 cesta (ref11) 2010
References_xml	– year: 2006 ident: ref19 article-title: Tutorial on planning activities for earth watching and observation satellites and constellations: From off-line ground planning to on-line on-board planning publication-title: Proc 16th Int Conf Automated Planning Scheduling – ident: ref22 doi: 10.1016/j.cja.2018.12.018 – year: 1998 ident: ref7 article-title: Sharing the use of a satellite: An overview of methods publication-title: Proc SpaceOps – ident: ref9 doi: 10.1007/978-3-642-26001-8_58 – ident: ref5 doi: 10.3390/s18051649 – start-page: 15 year: 2010 ident: ref11 article-title: Simulating on-board autonomy in a multi-agent system with planning and scheduling publication-title: Proc 20th Int Conf Planning Scheduling – volume: 10 start-page: 10703 year: 2014 ident: ref12 article-title: Research on task scheduling mechanism of distributed satellite system publication-title: J Comput Inf Syst – ident: ref3 doi: 10.1109/LGRS.2017.2702137 – ident: ref10 doi: 10.2514/6.1999-4479 – ident: ref16 doi: 10.1016/B978-0-934613-63-7.50009-7 – ident: ref14 doi: 10.1016/j.eswa.2009.07.079 – volume: 371 start-page: 16 year: 2014 ident: ref25 article-title: Coordinated multi-robot exploration under communication constraints using decentralized Markov decision processes publication-title: Philos Trans Roy Soc A Math Phys Eng Sci – ident: ref15 doi: 10.1016/j.actaastro.2018.11.001 – ident: ref13 doi: 10.1016/S0004-3702(02)00382-X – start-page: 989 year: 2014 ident: ref27 publication-title: Multi-agent Machine Learning A Reinforcement Approach – ident: ref28 doi: 10.1007/BF02460000 – year: 2017 ident: ref24 article-title: Multiagent bidirectionally-coordinated nets: Emergence of human-level coordination in learning to play StarCraft combat games publication-title: arXiv 1703 10069 – ident: ref21 doi: 10.2514/6.2018-2500 – ident: ref2 doi: 10.3390/s18041297 – volume: 20 start-page: 708 year: 2016 ident: ref1 article-title: The 'Internet plus' space-based information services publication-title: J Remote Sens – start-page: 941 year: 2013 ident: ref26 article-title: Multiagent reinforcement learning in a distributed sensor network with indirect feedback publication-title: Proc Int Conf Auton Agents and Multi Agent Syst – start-page: 6379 year: 2017 ident: ref30 article-title: Multi-agent actor-critic for mixed cooperative-competitive environments publication-title: Proc Adv Neural Inf Process Syst – volume: 1054 year: 2005 ident: ref23 publication-title: Reinforcement Learning An Introduction – year: 2018 ident: ref31 publication-title: Scalable Multi-Agent Reinforcement Learning for Cooperative Environment – year: 2011 ident: ref33 publication-title: Research on Imaging Satellite Mission Planning Technology – volume: 4 start-page: 5 year: 2001 ident: ref6 article-title: A review of technical development of military reconnaissance satellite abroad publication-title: Modern Defence Technology – start-page: 71 year: 2013 ident: ref17 article-title: Earth observing satellite cooperation missions scheduling based on feedback amendatory strategy publication-title: Proc Satellite Payload Technol Academic Annu Meeting – ident: ref29 doi: 10.1016/B978-1-55860-307-3.50049-6 – volume: 35 start-page: 2093 year: 2013 ident: ref8 article-title: Collaborative scheduling model and algorithm for image satellite network publication-title: Syst Eng Electron – ident: ref20 doi: 10.1109/ICNC.2015.7378139 – ident: ref4 doi: 10.3390/s17102347 – ident: ref18 doi: 10.1007/s10458-010-9145-2 – year: 2016 ident: ref32 article-title: A tour of TensorFlow publication-title: arXiv 1610 01178
SSID	ssj0024887
Score	2.5019631
Snippet	The provision of real-time information services is one of the crucial functions of satellites. In comparison with the centralized scheduling, the distributed...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	1901
SubjectTerms	Aerospace electronics Algorithms Communication Decisions Deep learning Deep reinforcement learning (DRL) distributed scheduling Games imaging satellites Information services Machine learning Markov processes Multiagent systems neural network Policies Real time real-time scheduling Real-time systems Reinforcement Response time Satellite observation Satellites Scheduling Task analysis Task scheduling
Title	An Online Distributed Satellite Cooperative Observation Scheduling Algorithm Based on Multiagent Deep Reinforcement Learning
URI	https://ieeexplore.ieee.org/document/9152114 https://www.proquest.com/docview/2586601060
Volume	18
WOSCitedRecordID	wos000711828000012&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Xplore / Electronic Library Online (IEL) customDbUrl: eissn: 1558-0571 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0024887 issn: 1545-598X databaseCode: RIE dateStart: 20040101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1BT9swFH7qEGhcGCubKGPIh53QMlI7jpNj1w44IEDthnqLkvi5IHVN1QYkJH78_By3ExqaxC2SbcnS55fnZ_v7PoAvBoWO0NjFm-REyeHdoKB4LOys7e40yhv3hpsLdXmZjMfpdQu-rrkwiOgen-E3-nR3-boq7-mo7CSlZEOu1W-Uihuu1l9dvcSZ4dGOIJBpMvY3mN0wPbk4G45sJchtgUrymVw8y0HOVOWfP7FLL6fvXjexXdjx20jWa3B_Dy2cteGtdzS_fWzD1pmz7H3cg6fejDWComxAKrlkcIWajXKnxVkj61fVHBsBcHZVrE9p2cjiqemh-oT1ppNqcVff_mbfbdbTzLY65m5OxCw2QJyzIToN1tIdNzIv2zr5AL9Of_zsnwfecyEohUzrgKi1RkktEXlJxZKKFZdFl2upI2PIJcPEqpDacNtS2HInERghzxON3Ugn4iNszKoZ7gNLbfyVZShyJVQUCyxyZZSwCyCPTBzLogPhCoWs9ILk5IsxzVxhEqYZAZcRcJkHrgPH6yHzRo3jf533CKl1Rw9SBw5XUGc-XpcZl0nsbpzDg5dHfYJtTq9ZHAvxEDbqxT1-hs3yob5bLo7cUvwDAxfbsw
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NT9wwEB0hKKIXaPlQt6XFB06IQNaO4-S4QPkQ2wWxgPYWJfF4QaKb1RIqIfXH1-OYRQhUiVsk25Kl58l4bL_3ADYNCh2hsYs3yYmSw9tBQfFY2Fnb3WmUN-4N113V6yWDQXo-A9tTLgwiusdnuEOf7i5fV-UDHZXtppRsyLV6jpyzPFvrWVkvcXZ4tCcIZJoM_B1mO0x3u0cXfVsLcluikoAmFy-ykLNVefUvdgnmcOl9U_sEi34jyToN8p9hBkfLsOA9zW8el2H-yJn2Pq7A386INZKi7IB0csniCjXr506Ns0a2X1VjbCTA2VkxPadlfYuopqfqQ9a5G1aT2_rmN9uzeU8z2-q4uzlRs9gB4phdoFNhLd2BI_PCrcNVuDr8ebl_HHjXhaAUMq0DItcaJbVE5CWVSypWXBZtrqWOjCGfDBOrQmrDbUthC55EYIQ8TzS2I52INZgdVSP8Aiy1EViWociVUFEssMiVUcIugTwycSyLFoRPKGSllyQnZ4y7zJUmYZoRcBkBl3ngWrA1HTJu9Dj-13mFkJp29CC1YP0J6sxH7H3GZRK7O-fw69ujNmDh-PJXN-ue9E6_wUdOb1scJ3EdZuvJA36HD-Wf-vZ-8sMty3-cOt78
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+Online+Distributed+Satellite+Cooperative+Observation+Scheduling+Algorithm+Based+on+Multiagent+Deep+Reinforcement+Learning&rft.jtitle=IEEE+geoscience+and+remote+sensing+letters&rft.au=Li%2C+Dalin&rft.au=Wang+Haijiao&rft.au=Yang%2C+Zhen&rft.au=Gu+Yanfeng&rft.date=2021-11-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=1545-598X&rft.eissn=1558-0571&rft.volume=18&rft.issue=11&rft.spage=1901&rft_id=info:doi/10.1109%2FLGRS.2020.3009823&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1545-598X&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1545-598X&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1545-598X&client=summon