RESPECT: Reinforcement Learning based Edge Scheduling on Pipelined Coral Edge TPUs
Deep neural networks (DNNs) have substantial computational and memory requirements, and the compilation of its computational graphs has a great impact on the performance of resource-constrained (e.g., computation, I/O, and memory-bound) edge computing systems. While efficient execution of their comp...
Uložené v:
| Vydané v: | 2023 60th ACM/IEEE Design Automation Conference (DAC) s. 1 - 6 |
|---|---|
| Hlavní autori: | , , , |
| Médium: | Konferenčný príspevok.. |
| Jazyk: | English |
| Vydavateľské údaje: |
IEEE
09.07.2023
|
| Predmet: | |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Deep neural networks (DNNs) have substantial computational and memory requirements, and the compilation of its computational graphs has a great impact on the performance of resource-constrained (e.g., computation, I/O, and memory-bound) edge computing systems. While efficient execution of their computational graph requires an effective scheduling algorithm, generating the optimal scheduling solution is a challenging NP-hard problem. Furthermore, the complexity of scheduling DNN computational graphs will further increase on pipelined multi-core systems considering memory communication cost, as well as the increasing size of DNNs. Using the synthetic graph for the training dataset, this work presents a reinforcement learning (RL) based scheduling framework RESPECT, which learns the behaviors of optimal optimization algorithms and generates near-optimal scheduling results with short solving runtime overhead. Our framework has demonstrated up to ∼ 2.5 × real-world on-chip inference runtime speedups over the commercial compiler with ten popular ImageNet models deployed on the physical Coral Edge TPUs system. Moreover, compared to the exact optimization methods, the proposed RL scheduling improves the scheduling optimization runtime by up to 683× speedups compared to the commercial compiler and matches the exact optimal solutions with up to 930× speedups. Finally, we perform a comprehensive generalizability test, which demonstrates RESPECT successfully imitates optimal solving behaviors from small synthetic graphs to large real-world DNNs computational graphs. |
|---|---|
| AbstractList | Deep neural networks (DNNs) have substantial computational and memory requirements, and the compilation of its computational graphs has a great impact on the performance of resource-constrained (e.g., computation, I/O, and memory-bound) edge computing systems. While efficient execution of their computational graph requires an effective scheduling algorithm, generating the optimal scheduling solution is a challenging NP-hard problem. Furthermore, the complexity of scheduling DNN computational graphs will further increase on pipelined multi-core systems considering memory communication cost, as well as the increasing size of DNNs. Using the synthetic graph for the training dataset, this work presents a reinforcement learning (RL) based scheduling framework RESPECT, which learns the behaviors of optimal optimization algorithms and generates near-optimal scheduling results with short solving runtime overhead. Our framework has demonstrated up to ∼ 2.5 × real-world on-chip inference runtime speedups over the commercial compiler with ten popular ImageNet models deployed on the physical Coral Edge TPUs system. Moreover, compared to the exact optimization methods, the proposed RL scheduling improves the scheduling optimization runtime by up to 683× speedups compared to the commercial compiler and matches the exact optimal solutions with up to 930× speedups. Finally, we perform a comprehensive generalizability test, which demonstrates RESPECT successfully imitates optimal solving behaviors from small synthetic graphs to large real-world DNNs computational graphs. |
| Author | Robinson, Daniel Yin, Jiaqi Yu, Cunxi Li, Yingjie |
| Author_xml | – sequence: 1 givenname: Jiaqi surname: Yin fullname: Yin, Jiaqi email: jiaqi.yin@utah.edu organization: University of Utah,Salt Lake City,US – sequence: 2 givenname: Yingjie surname: Li fullname: Li, Yingjie email: yingjie.li@utah.edu organization: University of Utah,Salt Lake City,US – sequence: 3 givenname: Daniel surname: Robinson fullname: Robinson, Daniel email: u0714849@umail.utah.edu organization: University of Utah,Salt Lake City,US – sequence: 4 givenname: Cunxi surname: Yu fullname: Yu, Cunxi email: cunxi.yu@utah.edu organization: University of Utah,Salt Lake City,US |
| BookMark | eNo1T9tKw0AUXEFBrfkDkfxA4t4vvpUYLxAwpOlz2d2c1IV0U5L64N8bqT7NMDMMM7foMo4REHogOCcEm8fndSGkoSanmLKcYMqVwvICJUYZzQRmlHFNrlEyz8FhiYXmWPIb1DTlpi6L9iltIMR-nDwcIJ7SCuwUQ9ynzs7QpWW3h3TjP6H7Gn7VMaZ1OMLCF7MYJzucI229ne_QVW-HGZI_XKHtS9kWb1n18fperKvMUoNPmV9mc0-16I3hxnnbmR4bbSwnVAglJPeOYAW91JJ0vJOEEiaUVk5Z57lgK3R_7g0AsDtO4WCn793_dfYDt4JPNw |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/DAC56929.2023.10247706 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9798350323481 |
| EndPage | 6 |
| ExternalDocumentID | 10247706 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Science Foundation funderid: 10.13039/100000001 |
| GroupedDBID | 6IE 6IH ACM ALMA_UNASSIGNED_HOLDINGS CBEJK RIE RIO |
| ID | FETCH-LOGICAL-a290t-c1094c285f9949bcad9f0989a412557564cb107ef6861d4d612135787b7abc453 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 4 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001073487300041&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:51:00 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a290t-c1094c285f9949bcad9f0989a412557564cb107ef6861d4d612135787b7abc453 |
| PageCount | 6 |
| ParticipantIDs | ieee_primary_10247706 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-July-9 |
| PublicationDateYYYYMMDD | 2023-07-09 |
| PublicationDate_xml | – month: 07 year: 2023 text: 2023-July-9 day: 09 |
| PublicationDecade | 2020 |
| PublicationTitle | 2023 60th ACM/IEEE Design Automation Conference (DAC) |
| PublicationTitleAbbrev | DAC |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssib060584064 |
| Score | 2.2723522 |
| Snippet | Deep neural networks (DNNs) have substantial computational and memory requirements, and the compilation of its computational graphs has a great impact on the... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 1 |
| SubjectTerms | Image edge detection Optimal scheduling Optimization methods Processor scheduling Reinforcement learning Runtime Training |
| Title | RESPECT: Reinforcement Learning based Edge Scheduling on Pipelined Coral Edge TPUs |
| URI | https://ieeexplore.ieee.org/document/10247706 |
| WOSCitedRecordID | wos001073487300041&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELagYmACRBFveWB1SdKLnWNDoRUDqqLSSt2q-JGqS1r1we-vz01BDAxslh3Lyl2s72Lf9x1jT0qDLrWxAqUDAdqA0JjFwodG1gfsUZq4IJn_oQaDbDLBoiGrBy6Mcy4kn7kONcNdvl2YLR2V-R2egFIksH2slNyTtQ4fD13veXCChgUcR_j89pqn0sN_h0qEdw6Tf5VRCSjSP_vn-ues_cPH48U30lywI1dfsqG3XdHLRy986IL8qQknfbxRTJ1xAijLe3bm-Kd3jaWc8xlf1LyYL4mF7gdz4ufvHxkV43Wbjfu9Uf4umgoJokww2gjjXxNMkqUVIqA2pcUqwgxL8HGLD8QkGO3_71wlMxlbsCQXRvI2SivvIEi7V6xVL2p3zThCZrpAwsWoqRyx1kZ2KxlZSNHFTt6wNhlkutyLYEwPtrj9o_-OnZLZQ2Yr3rPWZrV1D-zEfG3m69VjcN0OWsaXZw |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NT4NAEN2YaqInNdb47R68UoEOC-PNYJsaa0MqTXpr2A-aXmjTD3-_O1uq8eDBGwE2gRk289id9x5jD7EEWUilPRQGPJAKPIlJ4FlopC1g96PQOMn8fjwYJOMxZjVZ3XFhjDGu-cy06NDt5eu52tBSmZ3hIcQxCWzvk3VWTdfafT60wWfLE9Q84MDHx5fnNBIWALTIJLy1G_7LSMXVke7xP5_ghDV_GHk8-641p2zPVGdsaKOXddL8iQ-NE0BVbq2P15qpU04lSvOOnhr-YZOjqet8yucVz2YL4qHbiykx9Le35Nlo1WSjbidPe17tkeAVIfprT9nXBBUmUYkIKFWhsfQxwQIscrFQTICS9g_PlCIRgQZNgmEkcBPL2KYIovY5a1TzylwwjpCoNpB0MUoyJJZSiXYpfA0RmsCIS9akgEwWWxmMyS4WV3-cv2eHvfy9P-m_Dt6u2RGlwPW54g1rrJcbc8sO1Od6tlreuTR-AXMQmrA |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2023+60th+ACM%2FIEEE+Design+Automation+Conference+%28DAC%29&rft.atitle=RESPECT%3A+Reinforcement+Learning+based+Edge+Scheduling+on+Pipelined+Coral+Edge+TPUs&rft.au=Yin%2C+Jiaqi&rft.au=Li%2C+Yingjie&rft.au=Robinson%2C+Daniel&rft.au=Yu%2C+Cunxi&rft.date=2023-07-09&rft.pub=IEEE&rft.spage=1&rft.epage=6&rft_id=info:doi/10.1109%2FDAC56929.2023.10247706&rft.externalDocID=10247706 |