A heuristic fault tolerant MapReduce framework for minimizing makespan in Hybrid Cloud Environment

Cloud Computing propounds a striking option for business to pay only for the resources that were consumed. The prime challenge is to increase the MapReduce clusters to minimize their costs. MapReduce is a widely used parallel computing framework for large scale data processing. The major concern of...

Full description

Saved in:
Bibliographic Details
Published in:2014 International Conference on Green Computing Communication and Electrical Engineering (ICGCCEE) pp. 1 - 4
Main Authors: Raju, R., Amudhavel, J., Pavithra, M., Anuja, S., Abinaya, B.
Format: Conference Proceeding
Language:English
Published: IEEE 01.03.2014
Subjects:
Algorithm design and analysis
Cloud computing
Clustering algorithms
DAG
Dynamic scheduling
Hadoop
Heuristic algorithms
Johnson algorithm
Makespan
MapReduce
MPT
Processor scheduling
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Cloud Computing propounds a striking option for business to pay only for the resources that were consumed. The prime challenge is to increase the MapReduce clusters to minimize their costs. MapReduce is a widely used parallel computing framework for large scale data processing. The major concern of map reduce programming model are job execution time and cluster throughput. Multiple speculative execution strategies have been proposed, but all are failed to address the DAG communication and cluster utilization. In this paper, we developed a new strategy, OTA (Optimal Time Algorithm), which improves the effectiveness of speculative execution significantly. OTA do not consider the difference between the execution time of tasks on the same processors, they may form clusters of tasks that are not similar to each other. The proposed strategy efficiently utilizes the characteristics and properties of the MapReduce jobs in the given workload for constructing optimal job schedule. This resolves the problem of minimizing the makespan of workloads that additionally includes the workflow (DAGs) of mapreduce jobs.
DOI:10.1109/ICGCCEE.2014.6922462