DNN Inference Acceleration for Smart Devices in Industry 5.0 by Decentralized Deep Reinforcement Learning

With the emergence of Industry 5.0, there has been a significant surge in the need for intelligent services within the realm of smart devices. Currently, deep neural networks (DNNs) have become the predominant technology in driving advancements in intelligent applications. With the collaboration of...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE transactions on consumer electronics Ročník 70; číslo 1; s. 1519 - 1530
Hlavní autoři:	Dong, Chongwu, Shafiq, Muhammad, Dabel, Maryam M. Al, Sun, Yanbin, Tian, Zhihong
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	New York IEEE 01.02.2024 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:	Algorithms Artificial neural networks Computation offloading Decision making Deep learning Deep reinforcement learning DNN inference Edge AI Edge computing edge intelligence Electronic devices Fifth Industrial Revolution Industrial applications Industrial Internet of Things Industry 5.0 Inference Long short term memory Machine learning Markov processes Mobile computing multi-agent reinforcement learning Multi-agent systems Multiagent systems Network latency Optimization Performance evaluation Smart devices Synchronism
ISSN:	0098-3063, 1558-4127
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Abstract	With the emergence of Industry 5.0, there has been a significant surge in the need for intelligent services within the realm of smart devices. Currently, deep neural networks (DNNs) have become the predominant technology in driving advancements in intelligent applications. With the collaboration of mobile edge computing (MEC), resource-constraint smart devices, such as the industrial Internet of Things (IIoT) devices, can meet the requirement of high computing for DNN-based inference by computation offloading. In the task offloading strategy obtained by a central decision-maker with global information, all devices in the MEC can get the optimal optimization for DNN inference acceleration. However, in a practical environment, central decision-making may get into trouble, such as information synchronization delay, irrational behavior of devices, and privacy leakage. In this paper, we explore the optimization of distributed task offloading for smart devices to deal with these challenges regarding DNN inference acceleration, considering the character of an early exit in the DNN model to balance the accuracy and latency. In our system model, the optimization is formulated as a decentralized partially observable Markov decision process (Dec-POMDP). Each smart device performs its strategy, including task offloading decision and DNN branch selection with local observation, and cooperatively optimizes the overall Quality of Experience for DNN inference. Based on the model of Dec-POMDP, we propose one algorithm based on Multi-agent Reinforcement Learning to solve the above problem. In our algorithm, we utilize the advanced function based on the counterfactual baseline to guide policy gradient learning to overcome the credit allocation problem in cooperative optimization. In addition, LSTM is introduced to improve the robustness of the algorithm. Finally, detailed performance evaluation and comparison are performed to show the effectiveness of our strategy.
AbstractList	With the emergence of Industry 5.0, there has been a significant surge in the need for intelligent services within the realm of smart devices. Currently, deep neural networks (DNNs) have become the predominant technology in driving advancements in intelligent applications. With the collaboration of mobile edge computing (MEC), resource-constraint smart devices, such as the industrial Internet of Things (IIoT) devices, can meet the requirement of high computing for DNN-based inference by computation offloading. In the task offloading strategy obtained by a central decision-maker with global information, all devices in the MEC can get the optimal optimization for DNN inference acceleration. However, in a practical environment, central decision-making may get into trouble, such as information synchronization delay, irrational behavior of devices, and privacy leakage. In this paper, we explore the optimization of distributed task offloading for smart devices to deal with these challenges regarding DNN inference acceleration, considering the character of an early exit in the DNN model to balance the accuracy and latency. In our system model, the optimization is formulated as a decentralized partially observable Markov decision process (Dec-POMDP). Each smart device performs its strategy, including task offloading decision and DNN branch selection with local observation, and cooperatively optimizes the overall Quality of Experience for DNN inference. Based on the model of Dec-POMDP, we propose one algorithm based on Multi-agent Reinforcement Learning to solve the above problem. In our algorithm, we utilize the advanced function based on the counterfactual baseline to guide policy gradient learning to overcome the credit allocation problem in cooperative optimization. In addition, LSTM is introduced to improve the robustness of the algorithm. Finally, detailed performance evaluation and comparison are performed to show the effectiveness of our strategy.
Author	Sun, Yanbin Shafiq, Muhammad Dong, Chongwu Tian, Zhihong Dabel, Maryam M. Al
Author_xml	– sequence: 1 givenname: Chongwu orcidid: 0000-0002-9161-0570 surname: Dong fullname: Dong, Chongwu email: dcw650@gmail.com organization: Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou, China – sequence: 2 givenname: Muhammad orcidid: 0000-0003-1909-9373 surname: Shafiq fullname: Shafiq, Muhammad email: mshafiq@ieee.org organization: Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou, China – sequence: 3 givenname: Maryam M. Al orcidid: 0000-0003-4371-8939 surname: Dabel fullname: Dabel, Maryam M. Al email: maldabel@uhb.edu.sa organization: Department of Computer Science and Engineering, College of Computer Science and Engineering, University of Hafr Al Batin, Hafr Al Batin, Saudi Arabia – sequence: 4 givenname: Yanbin orcidid: 0000-0002-8157-5337 surname: Sun fullname: Sun, Yanbin email: sunyanbin@gzhu.edu.cn organization: Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou, China – sequence: 5 givenname: Zhihong orcidid: 0000-0002-9409-5359 surname: Tian fullname: Tian, Zhihong email: tianzhihong@gzhu.edu.cn organization: Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou, China
BookMark	eNp9kE1LAzEQQINUsFbvHjwEPG-dfOxHjqWtWigVtJ6XbHYiKW22ZrdC_fWmtgfx4CmEeS9h3iXp-cYjITcMhoyBul-Op0MOXAyFEEpmxRnpszQtEsl43iN9AFUkAjJxQS7bdgXAZMqLPnGTxYLOvMWA3iAdGYNrDLpzjae2CfR1o0NHJ_jpDLbU-cjWu7YLe5oOgVb7ODLou6DX7gvreMMtfUHno2twEyd0jjp459-vyLnV6xavT-eAvD1Ml-OnZP78OBuP5onhincJh1Sb2jKGxgJnqrBphqmVoGyFWoHMhKlAVwJqK1ktGKRM6SIvQCpUlRYDcnd8dxuajx22XblqdsHHL0sRGak4ZxCp7EiZ0LRtQFsa1_2sHXdx65JBechaxqzlIWt5yhpF-CNug4uR9v8pt0fFIeIvXEie57n4Bp1MhD0
CODEN	ITCEDA
CitedBy_id	crossref_primary_10_1016_j_est_2024_110574 crossref_primary_10_1109_TCE_2025_3566725 crossref_primary_10_1109_TII_2025_3567379 crossref_primary_10_1109_TSMC_2024_3454118 crossref_primary_10_3390_machines13090755 crossref_primary_10_3390_math13050754 crossref_primary_10_1109_TCE_2025_3536438 crossref_primary_10_1109_JIOT_2025_3585025
Cites_doi	10.1109/TCOMM.2023.3244958 10.1162/neco.1997.9.8.1735 10.1109/JPROC.2019.2918951 10.1016/j.jksuci.2023.02.013 10.1016/j.future.2022.10.033 10.1109/JIOT.2020.3002255 10.1049/cmu2.12334 10.1609/aaai.v35i11.17166 10.1109/TII.2020.3017573 10.1016/j.jmsy.2022.09.017 10.1109/COMST.2022.3199544 10.1109/TITS.2022.3230430 10.1109/TVT.2023.3250274 10.1145/3317572 10.1109/JIOT.2023.3246100 10.1007/s10489-022-04394-3 10.1109/TPDS.2021.3137631 10.1109/TCC.2022.3163750 10.1016/j.comnet.2021.107950 10.1109/icccs52626.2021.9449178 10.1109/ICPR.2016.7900006 10.1109/TII.2022.3206787 10.1109/TMC.2020.3025116 10.1109/TVT.2018.2799620 10.1145/3545008.3545071 10.1109/TVT.2023.3253508 10.1145/3517206.3526270 10.1145/3229556.3229562 10.1109/ICC45855.2022.9838879 10.1109/JIOT.2023.3279271 10.1109/JPROC.2022.3226481 10.1016/j.adhoc.2023.103193 10.1145/3555802 10.1109/JIOT.2020.2971323 10.1007/s10723-023-09659-w 10.1609/aaai.v32i1.11794 10.1109/TVT.2023.3236327 10.1016/j.cose.2020.101863 10.1109/ICIP46576.2022.9897574 10.1016/j.jpdc.2022.10.004 10.1109/TGCN.2021.3099804
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID	97E RIA RIE AAYXX CITATION 7SP 8FD F28 FR3 L7M
DOI	10.1109/TCE.2023.3339468
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Xplore Digital Library CrossRef Electronics & Communications Abstracts Technology Research Database ANTE: Abstracts in New Technology & Engineering Engineering Research Database Advanced Technologies Database with Aerospace
DatabaseTitle	CrossRef Engineering Research Database Technology Research Database Advanced Technologies Database with Aerospace ANTE: Abstracts in New Technology & Engineering Electronics & Communications Abstracts
DatabaseTitleList	Engineering Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1558-4127
EndPage	1530
ExternalDocumentID	10_1109_TCE_2023_3339468 10342777
Genre	orig-research
GrantInformation_xml	– fundername: National Natural Science Foundation of China grantid: 62250410365; 62272119; 62072130 funderid: 10.13039/501100001809 – fundername: National Key Research and Development Program of China grantid: 2021YFB3101704 funderid: 10.13039/501100010903
GroupedDBID	-~X .DC 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABJNI ABQJQ ABVLG ACGFO ACIWK ACKIV ACNCT AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 IBMZZ ICLAB IFIPE IFJZH IPLJI JAVBF LAI MS~ O9- OCL P2P RIA RIE RNS TAE TN5 VH1 AAYXX CITATION 7SP 8FD F28 FR3 L7M
ID	FETCH-LOGICAL-c292t-205acdf11ecf02198f56e5f409fbea90463cb0ab30df41d310519a878049e9ba3
IEDL.DBID	RIE
ISICitedReferencesCount	11
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001244904800426&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0098-3063
IngestDate	Tue Sep 23 15:10:48 EDT 2025 Sat Nov 29 01:45:44 EST 2025 Tue Nov 18 22:18:31 EST 2025 Wed Aug 27 01:52:56 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	1
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c292t-205acdf11ecf02198f56e5f409fbea90463cb0ab30df41d310519a878049e9ba3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0003-4371-8939 0000-0003-1909-9373 0000-0002-8157-5337 0000-0002-9161-0570 0000-0002-9409-5359
PQID	3049492210
PQPubID	85469
PageCount	12
ParticipantIDs	ieee_primary_10342777 crossref_primary_10_1109_TCE_2023_3339468 proquest_journals_3049492210 crossref_citationtrail_10_1109_TCE_2023_3339468
PublicationCentury	2000
PublicationDate	2024-02-01
PublicationDateYYYYMMDD	2024-02-01
PublicationDate_xml	– month: 02 year: 2024 text: 2024-02-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on consumer electronics
PublicationTitleAbbrev	T-CE
PublicationYear	2024
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 Hausknecht (ref41) ref12 ref34 ref15 ref37 ref14 ref31 ref30 ref11 ref33 ref10 Lowe (ref18) ref32 Wang (ref46) 2020 ref2 ref1 ref17 ref39 ref16 ref38 ref19 Krizhevsky (ref35); 25 ref24 ref23 ref45 ref26 ref25 ref20 ref42 ref22 Krizhevsky (ref36) 2009 ref44 ref21 ref43 ref28 ref27 ref29 ref8 ref7 ref9 ref4 ref3 ref6 ref5 ref40
References_xml	– ident: ref17 doi: 10.1109/TCOMM.2023.3244958 – ident: ref42 doi: 10.1162/neco.1997.9.8.1735 – ident: ref6 doi: 10.1109/JPROC.2019.2918951 – ident: ref20 doi: 10.1016/j.jksuci.2023.02.013 – ident: ref8 doi: 10.1016/j.future.2022.10.033 – ident: ref1 doi: 10.1109/JIOT.2020.3002255 – ident: ref30 doi: 10.1049/cmu2.12334 – ident: ref16 doi: 10.1609/aaai.v35i11.17166 – ident: ref34 doi: 10.1109/TII.2020.3017573 – ident: ref3 doi: 10.1016/j.jmsy.2022.09.017 – ident: ref25 doi: 10.1109/COMST.2022.3199544 – ident: ref33 doi: 10.1109/TITS.2022.3230430 – ident: ref40 doi: 10.1109/TVT.2023.3250274 – ident: ref32 doi: 10.1145/3317572 – ident: ref45 doi: 10.1109/JIOT.2023.3246100 – ident: ref22 doi: 10.1007/s10489-022-04394-3 – ident: ref23 doi: 10.1109/TPDS.2021.3137631 – ident: ref14 doi: 10.1109/TCC.2022.3163750 – ident: ref31 doi: 10.1016/j.comnet.2021.107950 – ident: ref37 doi: 10.1109/icccs52626.2021.9449178 – ident: ref15 doi: 10.1109/ICPR.2016.7900006 – ident: ref21 doi: 10.1109/TII.2022.3206787 – year: 2020 ident: ref46 article-title: R-MADDPG for partially observable environments and limited communication publication-title: arXiv:2002.06684 – ident: ref44 doi: 10.1109/TMC.2020.3025116 – ident: ref43 doi: 10.1109/TVT.2018.2799620 – ident: ref24 doi: 10.1145/3545008.3545071 – ident: ref29 doi: 10.1109/TVT.2023.3253508 – ident: ref10 doi: 10.1145/3517206.3526270 – ident: ref38 doi: 10.1145/3229556.3229562 – ident: ref12 doi: 10.1109/ICC45855.2022.9838879 – ident: ref27 doi: 10.1109/JIOT.2023.3279271 – ident: ref7 doi: 10.1109/JPROC.2022.3226481 – ident: ref39 doi: 10.1016/j.adhoc.2023.103193 – year: 2009 ident: ref36 article-title: Learning multiple layers of features from tiny images – volume: 25 start-page: 1097 volume-title: Proc. Adv. Neural Inf. Process. Syst. ident: ref35 article-title: ImageNet classification with deep convolutional neural networks – ident: ref5 doi: 10.1145/3555802 – ident: ref28 doi: 10.1109/JIOT.2020.2971323 – ident: ref26 doi: 10.1007/s10723-023-09659-w – ident: ref19 doi: 10.1609/aaai.v32i1.11794 – start-page: 29 volume-title: Proc. AAAI Fall Symposia ident: ref41 article-title: Deep recurrent Q-learning for partially observable MDPs – ident: ref11 doi: 10.1109/TVT.2023.3236327 – ident: ref4 doi: 10.1016/j.cose.2020.101863 – start-page: 6379 volume-title: Proc. Adv. Neural Inf. Process. Syst. 30th Annu. Conf. Neural Inf. Process. Syst. ident: ref18 article-title: Multiagent actor-critic for mixed cooperative-competitive environments – ident: ref9 doi: 10.1109/ICIP46576.2022.9897574 – ident: ref13 doi: 10.1016/j.jpdc.2022.10.004 – ident: ref2 doi: 10.1109/TGCN.2021.3099804
SSID	ssj0014528
Score	2.4551244
Snippet	With the emergence of Industry 5.0, there has been a significant surge in the need for intelligent services within the realm of smart devices. Currently, deep...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	1519
SubjectTerms	Algorithms Artificial neural networks Computation offloading Decision making Deep learning Deep reinforcement learning DNN inference Edge AI Edge computing edge intelligence Electronic devices Fifth Industrial Revolution Industrial applications Industrial Internet of Things Industry 5.0 Inference Long short term memory Machine learning Markov processes Mobile computing multi-agent reinforcement learning Multi-agent systems Multiagent systems Network latency Optimization Performance evaluation Smart devices Synchronism
Title	DNN Inference Acceleration for Smart Devices in Industry 5.0 by Decentralized Deep Reinforcement Learning
URI	https://ieeexplore.ieee.org/document/10342777 https://www.proquest.com/docview/3049492210
Volume	70
WOSCitedRecordID	wos001244904800426&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-4127 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014528 issn: 0098-3063 databaseCode: RIE dateStart: 19750101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFA9ueNCDnxOnU3Lw4qFbP9KmOY59oJchOmG30qYvMtBu7EOYf70vaToGouCtpUkp_SV5v-S993uE3CGjzTylAiN06zDmp46QSjng5kpkaAMjcE2xCT4axZOJeLLJ6iYXBgBM8Bm09aXx5eczudZHZTjDA-ZzzmukxnlUJmttXQYs9ONKIBN5cFD5JF3RGfcGbV0mvB0EgWBaVXXHBpmiKj9WYmNehsf__LATcmR5JO2WwJ-SPSjOyOGOuuA5mfZHI_pYJfTRrpRoYkrAKVJV-vKBw4b2wawVdFpQW8ZjQ8O2S7MNPrKhm9MvyPEO5vQZjNKqNIeK1IqzvjXI63Aw7j04trKCI33hr3BqhKnMleeBVGjkRazCCEKFez2VQSq0ipjM3DQLEDHm5UgBkeilsRYrEiCyNLgg9WJWwCWhqZf7Oa4Mfgw545ErJEvDSCLxyGIugTdJp_rXibSy47r6xXtith-uSBCdRKOTWHSa5H7bY15KbvzRtqHR2GlXAtEkrQrPxE7KZaI9ikz4uMm9-qXbNTnAt7MyKrtF6qvFGm7IvvxcTZeLWzPevgHIaNFz
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFD7oFNQHrxPnNQ---NCtl3RtHkUnilpEJ-yttOmJFHQbugn66z1JUxmIgm8tTdrQL8n5kpPzHYBjYrS5p1RghG4dzv3MEVIpB91CiZxsYBddk2wiSpJ4MBB3NljdxMIgojl8hm19aXz5xUhO9VYZjfCA-1EUzcNCSK91q3Ctb6cBD_24lsgkJhzUXklXdPpnvbZOFN4OgkBwras6Y4VMWpUfc7ExMBdr_2zaOqxaJslOK-g3YA6Hm7Ayoy-4BeV5krCrOqSPnUpJRqaCnBFZZQ8v1HHYOZrZgpVDZhN5fLCw7bL8gx7Zw5vlJxZ0h2N2j0ZrVZptRWblWZ-a8HjR659dOja3giN94U9ocISZLJTnoVRk5kWswi6GilZ7KsdMaB0xmbtZHhBm3CuIBBLVy2ItVyRQ5FmwDY3haIg7wDKv8AuaG_wYCx51XSF5FnYlUY88jiRGLejU_zqVVnhc5794Ts0CxBUpoZNqdFKLTgtOvmuMK9GNP8o2NRoz5SogWrBf45naYfmWap8iFz4tc3d_qXYES5f925v05iq53oNl-hKvzmjvQ2PyOsUDWJTvk_Lt9dD0vS_X49S6
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=DNN+Inference+Acceleration+for+Smart+Devices+in+Industry+5.0+by+Decentralized+Deep+Reinforcement+Learning&rft.jtitle=IEEE+transactions+on+consumer+electronics&rft.au=Dong%2C+Chongwu&rft.au=Shafiq%2C+Muhammad&rft.au=Dabel%2C+Maryam+M.+Al&rft.au=Sun%2C+Yanbin&rft.date=2024-02-01&rft.issn=0098-3063&rft.eissn=1558-4127&rft.volume=70&rft.issue=1&rft.spage=1519&rft.epage=1530&rft_id=info:doi/10.1109%2FTCE.2023.3339468&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TCE_2023_3339468
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0098-3063&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0098-3063&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0098-3063&client=summon