Throughput Maximization of Delay-Aware DNN Inference in Edge Computing by Exploring DNN Model Partitioning and Inference Parallelism

Mobile Edge Computing (MEC) has emerged as a promising paradigm catering to overwhelming explosions of mobile applications, by offloading compute-intensive tasks to MEC networks for processing. The surging of deep learning brings new vigor and vitality to shape the prospect of intelligent Internet o...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on mobile computing Vol. 22; no. 5; pp. 3017 - 3030
Main Authors: Li, Jing, Liang, Weifa, Li, Yuchen, Xu, Zichuan, Jia, Xiaohua, Guo, Song
Format: Magazine Article
Language:English
Published: Los Alamitos IEEE 01.05.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
algorithm design and analysis
Algorithms
Applications programs
Approximation algorithms
approximation and online algorithms
Artificial neural networks
Computational modeling
computing and bandwidth resource allocation and optimization
Delay
delay-aware DNN inference
Delays
DNN model inference provisioning
DNN partitioning
Edge computing
Explosions
Inference
Inference algorithms
inference parallelism
Intelligent IoT devices
Internet of Things
Machine learning
Maximization
Mobile computing
Mobile edge computing (MEC)
Optimization
Parallel processing
Partitioning
Partitioning algorithms
Task analysis
throughput maximization
ISSN:1536-1233, 1558-0660
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Mobile Edge Computing (MEC) has emerged as a promising paradigm catering to overwhelming explosions of mobile applications, by offloading compute-intensive tasks to MEC networks for processing. The surging of deep learning brings new vigor and vitality to shape the prospect of intelligent Internet of Things (IoT), and edge intelligence arises to provision real-time deep neural network (DNN) inference services for users. To accelerate the processing of the DNN inference of a user request in an MEC network, the DNN inference model usually can be partitioned into two connected parts: one part is processed in the local IoT device of the request, and another part is processed in a cloudlet (edge server) in the MEC network. Also, the DNN inference can be further accelerated by allocating multiple threads of the cloudlet to which the request is assigned. In this paper, we study a novel delay-aware DNN inference throughput maximization problem with the aim to maximize the number of delay-aware DNN service requests admitted, by accelerating each DNN inference through jointly exploring DNN partitioning and multi-thread execution parallelism. Specifically, we consider the problem under both offline and online request arrival settings: a set of DNN inference requests is given in advance, and a sequence of DNN inference requests arrives one by one without the knowledge of future arrivals, respectively. We first show that the defined problems are NP-hard. We then devise a novel constant approximation algorithm for the problem under the offline setting. We also propose an online algorithm with a provable competitive ratio for the problem under the online setting. We finally evaluate the performance of the proposed algorithms through experimental simulations. Experimental results demonstrate that the proposed algorithms are promising
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1536-1233
1558-0660
DOI:10.1109/TMC.2021.3125949