A heuristic fault tolerant MapReduce framework for minimizing makespan in Hybrid Cloud Environment

Cloud Computing propounds a striking option for business to pay only for the resources that were consumed. The prime challenge is to increase the MapReduce clusters to minimize their costs. MapReduce is a widely used parallel computing framework for large scale data processing. The major concern of...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2014 International Conference on Green Computing Communication and Electrical Engineering (ICGCCEE) s. 1 - 4
Hlavní autori: Raju, R., Amudhavel, J., Pavithra, M., Anuja, S., Abinaya, B.
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 01.03.2014
Predmet:
Algorithm design and analysis
Cloud computing
Clustering algorithms
DAG
Dynamic scheduling
Hadoop
Heuristic algorithms
Johnson algorithm
Makespan
MapReduce
MPT
Processor scheduling
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Cloud Computing propounds a striking option for business to pay only for the resources that were consumed. The prime challenge is to increase the MapReduce clusters to minimize their costs. MapReduce is a widely used parallel computing framework for large scale data processing. The major concern of map reduce programming model are job execution time and cluster throughput. Multiple speculative execution strategies have been proposed, but all are failed to address the DAG communication and cluster utilization. In this paper, we developed a new strategy, OTA (Optimal Time Algorithm), which improves the effectiveness of speculative execution significantly. OTA do not consider the difference between the execution time of tasks on the same processors, they may form clusters of tasks that are not similar to each other. The proposed strategy efficiently utilizes the characteristics and properties of the MapReduce jobs in the given workload for constructing optimal job schedule. This resolves the problem of minimizing the makespan of workloads that additionally includes the workflow (DAGs) of mapreduce jobs.
DOI:10.1109/ICGCCEE.2014.6922462