An Optimal Locality-Aware Task Scheduling Algorithm Based on Bipartite Graph Modelling for Spark Applications

In the distributed computing framework of Spark, cross-node/rack data transfer produced by map tasks and reduce tasks are common problems resulting in performance degradation, such as prolonging of entire execution time and network congestion. To address these problems, this article utilizes the bip...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE transactions on parallel and distributed systems Vol. 31; no. 10; pp. 2406 - 2420
Main Authors:	Fu, Zhongming, Tang, Zhuo, Yang, Li, Liu, Chubo
Format:	Journal Article
Language:	English
Published:	New York IEEE 01.10.2020 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:	Algorithms Benchmarks Communication Communication cost Communications traffic Computer networks data locality Data transfer Data transfer (computers) Distributed processing Graph theory Modelling Optimal scheduling Optimization Performance degradation Performance evaluation Scheduling Scheduling algorithms Spark Sparks Task analysis Task scheduling weighted bipartite graph
ISSN:	1045-9219, 1558-2183
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Be the first to leave a comment!