Dynamic Split Computing Framework in Distributed Serverless Edge Clouds

Distributed serverless edge clouds and split computing are promising technologies to reduce the inference latency of large-scale deep neural networks (DNNs). In this article, we propose a dynamic split computing framework (DSCF) in distributed serverless edge clouds. In DSCF, the edge cloud orchestr...

Full description

Saved in:
Bibliographic Details
Published in:IEEE internet of things journal Vol. 11; no. 8; pp. 14523 - 14531
Main Authors: Ko, Haneul, Jeong, Hyeonjae, Jung, Daeyoung, Pack, Sangheon
Format: Journal Article
Language:English
Published: Piscataway IEEE 15.04.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Artificial neural networks
Cloud computing
Computational modeling
Consumption
Containers
Distributed serverless edge cloud
Edge computing
Inference
joint optimization
Linear programming
Markov processes
Mobile handsets
Predictive models
split computing
warm start
ISSN:2327-4662, 2327-4662
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Distributed serverless edge clouds and split computing are promising technologies to reduce the inference latency of large-scale deep neural networks (DNNs). In this article, we propose a dynamic split computing framework (DSCF) in distributed serverless edge clouds. In DSCF, the edge cloud orchestrator dynamically determines 1) splitting point and 2) warm status maintenance of container instances (i.e., whether or not to maintain each container instance in a warm status). For optimal decisions, we formulate a constrained Markov decision process (CMDP) problem to minimize the inference latency while maintaining the average resource consumption of distributed edge clouds below a certain level. The optimal stochastic policy can be obtained by converting the CMDP model into a linear programming (LP) model. The evaluation results demonstrate that DSCF can achieve less than half the inference latency compared to the local computing scheme while maintaining sufficient low resource consumption of distributed edge clouds.
AbstractList Distributed serverless edge clouds and split computing are promising technologies to reduce the inference latency of large-scale deep neural networks (DNNs). In this article, we propose a dynamic split computing framework (DSCF) in distributed serverless edge clouds. In DSCF, the edge cloud orchestrator dynamically determines 1) splitting point and 2) warm status maintenance of container instances (i.e., whether or not to maintain each container instance in a warm status). For optimal decisions, we formulate a constrained Markov decision process (CMDP) problem to minimize the inference latency while maintaining the average resource consumption of distributed edge clouds below a certain level. The optimal stochastic policy can be obtained by converting the CMDP model into a linear programming (LP) model. The evaluation results demonstrate that DSCF can achieve less than half the inference latency compared to the local computing scheme while maintaining sufficient low resource consumption of distributed edge clouds.
Author Pack, Sangheon
Ko, Haneul
Jeong, Hyeonjae
Jung, Daeyoung
Author_xml – sequence: 1
  givenname: Haneul
  orcidid: 0000-0002-9067-445X
  surname: Ko
  fullname: Ko, Haneul
  email: heko@khu.ac.kr
  organization: Department of Electronic Engineering, Kyung Hee University, Yongin, South Korea
– sequence: 2
  givenname: Hyeonjae
  surname: Jeong
  fullname: Jeong, Hyeonjae
  email: qeqe@korea.ac.kr
  organization: School of Electrical Engineering, Korea University, Seoul, South Korea
– sequence: 3
  givenname: Daeyoung
  surname: Jung
  fullname: Jung, Daeyoung
  email: eodud1884@korea.ac.kr
  organization: School of Electrical Engineering, Korea University, Seoul, South Korea
– sequence: 4
  givenname: Sangheon
  orcidid: 0000-0002-1085-1568
  surname: Pack
  fullname: Pack, Sangheon
  email: shpack@korea.ac.kr
  organization: School of Electrical Engineering, Korea University, Seoul, South Korea
BookMark eNp9kMtOwzAQRS1UJErpByCxiMQ6xY8kTpcofVBUqYuWdeTY48olL2wH1L8nVbuoWLCaWdxzZ3Tu0aBuakDokeAJIXj68r7a7CYUUzZhLKIRS2_QkDLKwyhJ6OBqv0Nj5w4Y4x6LyTQZouXsWIvKyGDblsYHWVO1nTf1PlhYUcFPYz8DUwcz47w1RedBBVuw32BLcC6Yqz0EWdl0yj2gWy1KB-PLHKGPxXyXvYXrzXKVva5DSaeRDyFNcJQyrUBpGXPGE66xIhAnUmvNgMQx11IrJUXBhQBNFHChC1zoVGOp2Qg9n3tb23x14Hx-aDpb9ydzhllMecQ57VP8nJK2cc6CzqXxwpum9laYMic4P4nLT-Lyk7j8Iq4nyR-ytaYS9vgv83RmDABc5VmcJP1Tv7VyfNs
CODEN IITJAU
CitedBy_id crossref_primary_10_1109_JIOT_2024_3415659
crossref_primary_10_1109_TWC_2024_3523517
crossref_primary_10_1109_TCC_2025_3591482
crossref_primary_10_1109_MIC_2024_3514670
crossref_primary_10_1109_JIOT_2024_3488283
Cites_doi 10.1109/JIOT.2020.3010258
10.1109/ICDCSW.2017.36
10.1109/JIOT.2020.2971323
10.1109/twc.2022.3165824
10.1109/TPDS.2022.3220221
10.1109/JIOT.2020.2981338
10.1145/3445814.3446757
10.1109/TPDS.2020.3032443
10.3390/s21134494
10.1109/TVT.2020.3039514
10.1109/JIOT.2018.2797174
10.1109/JIOT.2020.3004223
10.1145/3229556.3229562
10.1145/3372224.3419194
10.1109/TMC.2021.3136611
10.1145/3093336.3037698
10.1109/TNSM.2020.3023305
10.1145/3447548.3467078
10.1109/TMC.2019.2947893
10.1145/3527155
10.1109/JSAC.2021.3118403
10.1109/JSEN.2019.2939182
10.3390/s21165653
10.1109/JIOT.2021.3137291
10.1109/JPROC.2017.2761740
10.1109/PIMRC50174.2021.9569707
10.1109/MWC.001.2000466
10.1109/TSC.2021.3052139
10.5594/jmi.2021.3064082
10.1145/3338840.3355669
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
8FD
JQ2
L7M
L~C
L~D
DOI 10.1109/JIOT.2023.3342438
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE/IET Electronic Library (IEL) (UW System Shared)
CrossRef
Computer and Information Systems Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Computer and Information Systems Abstracts
Technology Research Database
Computer and Information Systems Abstracts – Academic
Advanced Technologies Database with Aerospace
ProQuest Computer Science Collection
Computer and Information Systems Abstracts Professional
DatabaseTitleList
Computer and Information Systems Abstracts
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE/IET Electronic Library (IEL) (UW System Shared)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 2327-4662
EndPage 14531
ExternalDocumentID 10_1109_JIOT_2023_3342438
10356630
Genre orig-research
GrantInformation_xml – fundername: Institute for Information and Communications Technology Planning and Evaluation (IITP)
  grantid: 2022-0-01015; 2022-0-00531
  funderid: 10.13039/501100010418
GroupedDBID 0R~
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABJNI
ABQJQ
ABVLG
AGQYO
AHBIQ
AKJIK
AKQYR
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
EBS
IFIPE
IPLJI
JAVBF
M43
OCL
PQQKQ
RIA
RIE
AAYXX
CITATION
7SC
8FD
JQ2
L7M
L~C
L~D
ID FETCH-LOGICAL-c294t-e860483fdedfc573767f0d1e56cfff3e1557fcfddcab7aaef1de7afb0bf8f0cf3
IEDL.DBID RIE
ISICitedReferencesCount 5
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001203466500021&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 2327-4662
IngestDate Mon Jun 30 05:06:05 EDT 2025
Tue Nov 18 22:42:22 EST 2025
Sat Nov 29 01:43:58 EST 2025
Wed Aug 27 01:53:53 EDT 2025
IsPeerReviewed false
IsScholarly true
Issue 8
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c294t-e860483fdedfc573767f0d1e56cfff3e1557fcfddcab7aaef1de7afb0bf8f0cf3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0002-1085-1568
0000-0002-9067-445X
PQID 3035274772
PQPubID 2040421
PageCount 9
ParticipantIDs proquest_journals_3035274772
crossref_citationtrail_10_1109_JIOT_2023_3342438
crossref_primary_10_1109_JIOT_2023_3342438
ieee_primary_10356630
PublicationCentury 2000
PublicationDate 2024-04-15
PublicationDateYYYYMMDD 2024-04-15
PublicationDate_xml – month: 04
  year: 2024
  text: 2024-04-15
  day: 15
PublicationDecade 2020
PublicationPlace Piscataway
PublicationPlace_xml – name: Piscataway
PublicationTitle IEEE internet of things journal
PublicationTitleAbbrev JIoT
PublicationYear 2024
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref34
ref15
ref14
ref31
ref30
ref11
ref10
ref32
ref2
ref1
ref17
ref16
Shilkov (ref33) 2021
ref19
ref18
(ref12) 2022
Hendrickson (ref7)
ref23
ref26
ref25
ref20
(ref24) 2023
ref22
ref21
ref28
ref27
ref29
ref8
ref9
ref4
ref3
ref6
ref5
References_xml – year: 2022
  ident: ref12
  article-title: Study on 5G system support for AI/ML-based services, Version 1.0.0
– ident: ref17
  doi: 10.1109/JIOT.2020.3010258
– ident: ref25
  doi: 10.1109/ICDCSW.2017.36
– ident: ref10
  doi: 10.1109/JIOT.2020.2971323
– ident: ref14
  doi: 10.1109/twc.2022.3165824
– volume-title: Comparison of cold starts in serverless functions across AWS, Azure, and GCP
  year: 2021
  ident: ref33
– ident: ref2
  doi: 10.1109/TPDS.2022.3220221
– ident: ref16
  doi: 10.1109/JIOT.2020.2981338
– start-page: 33
  volume-title: Proc. USENIX HotCloud
  ident: ref7
  article-title: Serverless computation with OpenLambda
– ident: ref26
  doi: 10.1145/3445814.3446757
– ident: ref31
  doi: 10.1109/TPDS.2020.3032443
– ident: ref23
  doi: 10.3390/s21134494
– ident: ref18
  doi: 10.1109/TVT.2020.3039514
– ident: ref30
  doi: 10.1109/JIOT.2018.2797174
– ident: ref9
  doi: 10.1109/JIOT.2020.3004223
– ident: ref19
  doi: 10.1145/3229556.3229562
– ident: ref20
  doi: 10.1145/3372224.3419194
– ident: ref29
  doi: 10.1109/TMC.2021.3136611
– ident: ref13
  doi: 10.1145/3093336.3037698
– ident: ref5
  doi: 10.1109/TNSM.2020.3023305
– ident: ref21
  doi: 10.1145/3447548.3467078
– ident: ref15
  doi: 10.1109/TMC.2019.2947893
– ident: ref3
  doi: 10.1145/3527155
– ident: ref4
  doi: 10.1109/JSAC.2021.3118403
– ident: ref6
  doi: 10.1109/JSEN.2019.2939182
– ident: ref34
  doi: 10.3390/s21165653
– ident: ref8
  doi: 10.1109/JIOT.2021.3137291
– ident: ref1
  doi: 10.1109/JPROC.2017.2761740
– ident: ref22
  doi: 10.1109/PIMRC50174.2021.9569707
– ident: ref27
  doi: 10.1109/MWC.001.2000466
– ident: ref28
  doi: 10.1109/TSC.2021.3052139
– volume-title: AWS Lambda Documentation
  year: 2023
  ident: ref24
– ident: ref11
  doi: 10.5594/jmi.2021.3064082
– ident: ref32
  doi: 10.1145/3338840.3355669
SSID ssj0001105196
Score 2.3361735
Snippet Distributed serverless edge clouds and split computing are promising technologies to reduce the inference latency of large-scale deep neural networks (DNNs)....
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 14523
SubjectTerms Artificial neural networks
Cloud computing
Computational modeling
Consumption
Containers
Distributed serverless edge cloud
Edge computing
Inference
joint optimization
Linear programming
Markov processes
Mobile handsets
Predictive models
split computing
warm start
Title Dynamic Split Computing Framework in Distributed Serverless Edge Clouds
URI https://ieeexplore.ieee.org/document/10356630
https://www.proquest.com/docview/3035274772
Volume 11
WOSCitedRecordID wos001203466500021&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE/IET Electronic Library (IEL) (UW System Shared)
  customDbUrl:
  eissn: 2327-4662
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0001105196
  issn: 2327-4662
  databaseCode: RIE
  dateStart: 20140101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT8MwDI5g4sCF8RhiMFAOnJA60lfSHtEePA4DiSHtVjWJjSahDa0dv58kbWESAolbDnFb2bVjx4-PkEsWpEGuAt8Tvo49Y_DAS7XgZqWSMAp8FKgc2ISYTJLZLH2qm9VdLwwAuOIz6Nuly-XrpVrbqzKj4aHxPkIToW8Lwatmre8LFd96I7zOXPosvX64f5z2LTx4PzSvjWwLysbZ48BUflhgd6yM2__8oH2yV_uP9KYS-AHZgsUhaTfYDLRW1SNyO6yg5umz8TJLWm0wxxQdN9VYdL6gQzs21yJegabWaoBNvRd0pF-BDt6Wa110yMt4NB3ceTVogqeCNCo9SLidEo8aNKpY2GEtyLQPMVeIGILxHwz7UWuVS5HngL4GkaNkEhNkCsNj0losF3BCKIRRlMtYS0MfccVTiaCjWCKTUiBnXcIadmaqnihugS3eMhdZsDSzEsisBLJaAl1y9UXyXo3T-Gtzx7J8Y2PF7S7pNULLao0rstAOdjWxkQhOfyE7I7vm6a7sxo97pFWu1nBOdtRHOS9WF-5n-gQbRMo9
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA6igl58VqxWzcGTsDXZV7pHqdbWRxWs0NuySWZEkFZs6-83k92qIArecpghS2aTmWRmvo-xYxFmYWFCGShpk8AdeBBkVqVuZFpRHEpUaDzZhOr3W8Nhdl81q_teGADwxWfQpKHP5duxmdFTmdvhkYs-IndDXyLqLFm2a309qUiKR9IqdylFdnrVuxs0iSC8GbmJY2pC-eZ9PJ3KjzPYO5bO-j8_aYOtVREkPytNvskWYLTF1ufsDLzarNvs8rwkm-cPLs6c8lLAOSremddj8ecRPyfgXOK8Asvp3ABKvk_4hX0C3n4Zz-ykxh47F4N2N6hoEwITZvE0gFZKOPFowaJJFMG1oLASktQgYgQugnAGQGtNoVVRAEoLqkAtNLZQGIx22OJoPIJdxiGK40InVjv9ODVpphFsnGgUWitMRZ2J-XLmpsIUJ2qLl9zfLUSWkwVyskBeWaDOTj5VXktAjb-Ea7Tk3wTL1a6zxtxoebXnJnlE0K7udqTCvV_UjthKd3B7k9_0-tf7bNXN5ItwZNJgi9O3GRywZfM-fZ68Hfof6wPF6s2E
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Dynamic+Split+Computing+Framework+in+Distributed+Serverless+Edge+Clouds&rft.jtitle=IEEE+internet+of+things+journal&rft.au=Ko%2C+Haneul&rft.au=Jeong%2C+Hyeonjae&rft.au=Jung%2C+Daeyoung&rft.au=Pack%2C+Sangheon&rft.date=2024-04-15&rft.pub=IEEE&rft.eissn=2327-4662&rft.volume=11&rft.issue=8&rft.spage=14523&rft.epage=14531&rft_id=info:doi/10.1109%2FJIOT.2023.3342438&rft.externalDocID=10356630
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2327-4662&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2327-4662&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2327-4662&client=summon