Performance Modelling and Cost Effective Execution for Distributed Graph Processing on Configurable VMs

Graph Processing has been widely used to capture complex data dependency and uncover relationship insights. Due to the ever-growing graph scale and algorithm complexity, distributed graph processing has become more and more popular. In this paper, we investigate how to balance performance and cost f...

Full description

Saved in:
Bibliographic Details
Published in:2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) pp. 74 - 83
Main Authors: Li, Zengxiang, Zhang, Bowen, Ren, Shen, Liu, Yong, Qin, Zheng, Goh, Rick Siow Mong, Gurusamy, Mohan
Format: Conference Proceeding
Language:English
Published: Piscataway, NJ, USA IEEE Press 14.05.2017
IEEE
Series:ACM Conferences
Subjects:
Complexity theory
Computational modeling
Cost Effective Execution
Distributed Graph Processing
Global Positioning System
Message systems
Performance Modelling and Prediction
Prediction algorithms
Predictive models
Resource Virtualization and Provisioning
Synchronization
Resource Virtualization and Provisioning
Performance Modelling and Prediction
Cost Effective Execution
Distributed Graph Processing
ISBN:9781509066100, 1509066101
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Graph Processing has been widely used to capture complex data dependency and uncover relationship insights. Due to the ever-growing graph scale and algorithm complexity, distributed graph processing has become more and more popular. In this paper, we investigate how to balance performance and cost for large scale graph processing on configurable virtual machines (VMs). We analyze the system architecture and implementation details of a Pregel-like distributed graph processing framework and develop a system-aware model to predict the execution time. Consequently, cost effective execution scenarios are recommended by selecting a certain number of VMs with specified capability subject to the predefined resource price and user preference. Experiments using synthetic and real world graphs have verified that system-aware model can achieve much higher prediction accuracy than popular machine-learning models which treat graph processing framework as a black box. As a result, the recommended execution scenarios have comparable cost efficiency to the optimal scenarios.
ISBN:9781509066100
1509066101
DOI:10.1109/CCGRID.2017.85