Dynamic Split Computing Framework in Distributed Serverless Edge Clouds

Distributed serverless edge clouds and split computing are promising technologies to reduce the inference latency of large-scale deep neural networks (DNNs). In this article, we propose a dynamic split computing framework (DSCF) in distributed serverless edge clouds. In DSCF, the edge cloud orchestr...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE internet of things journal Jg. 11; H. 8; S. 14523 - 14531
Hauptverfasser: Ko, Haneul, Jeong, Hyeonjae, Jung, Daeyoung, Pack, Sangheon
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Piscataway IEEE 15.04.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Artificial neural networks
Cloud computing
Computational modeling
Consumption
Containers
Distributed serverless edge cloud
Edge computing
Inference
joint optimization
Linear programming
Markov processes
Mobile handsets
Predictive models
split computing
warm start
ISSN:2327-4662, 2327-4662
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Distributed serverless edge clouds and split computing are promising technologies to reduce the inference latency of large-scale deep neural networks (DNNs). In this article, we propose a dynamic split computing framework (DSCF) in distributed serverless edge clouds. In DSCF, the edge cloud orchestrator dynamically determines 1) splitting point and 2) warm status maintenance of container instances (i.e., whether or not to maintain each container instance in a warm status). For optimal decisions, we formulate a constrained Markov decision process (CMDP) problem to minimize the inference latency while maintaining the average resource consumption of distributed edge clouds below a certain level. The optimal stochastic policy can be obtained by converting the CMDP model into a linear programming (LP) model. The evaluation results demonstrate that DSCF can achieve less than half the inference latency compared to the local computing scheme while maintaining sufficient low resource consumption of distributed edge clouds.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2327-4662
2327-4662
DOI:10.1109/JIOT.2023.3342438