A heuristic fault tolerant MapReduce framework for minimizing makespan in Hybrid Cloud Environment

Cloud Computing propounds a striking option for business to pay only for the resources that were consumed. The prime challenge is to increase the MapReduce clusters to minimize their costs. MapReduce is a widely used parallel computing framework for large scale data processing. The major concern of...

Full description

Saved in:
Bibliographic Details
Published in:2014 International Conference on Green Computing Communication and Electrical Engineering (ICGCCEE) pp. 1 - 4
Main Authors: Raju, R., Amudhavel, J., Pavithra, M., Anuja, S., Abinaya, B.
Format: Conference Proceeding
Language:English
Published: IEEE 01.03.2014
Subjects:
Algorithm design and analysis
Cloud computing
Clustering algorithms
DAG
Dynamic scheduling
Hadoop
Heuristic algorithms
Johnson algorithm
Makespan
MapReduce
MPT
Processor scheduling
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Cloud Computing propounds a striking option for business to pay only for the resources that were consumed. The prime challenge is to increase the MapReduce clusters to minimize their costs. MapReduce is a widely used parallel computing framework for large scale data processing. The major concern of map reduce programming model are job execution time and cluster throughput. Multiple speculative execution strategies have been proposed, but all are failed to address the DAG communication and cluster utilization. In this paper, we developed a new strategy, OTA (Optimal Time Algorithm), which improves the effectiveness of speculative execution significantly. OTA do not consider the difference between the execution time of tasks on the same processors, they may form clusters of tasks that are not similar to each other. The proposed strategy efficiently utilizes the characteristics and properties of the MapReduce jobs in the given workload for constructing optimal job schedule. This resolves the problem of minimizing the makespan of workloads that additionally includes the workflow (DAGs) of mapreduce jobs.
AbstractList Cloud Computing propounds a striking option for business to pay only for the resources that were consumed. The prime challenge is to increase the MapReduce clusters to minimize their costs. MapReduce is a widely used parallel computing framework for large scale data processing. The major concern of map reduce programming model are job execution time and cluster throughput. Multiple speculative execution strategies have been proposed, but all are failed to address the DAG communication and cluster utilization. In this paper, we developed a new strategy, OTA (Optimal Time Algorithm), which improves the effectiveness of speculative execution significantly. OTA do not consider the difference between the execution time of tasks on the same processors, they may form clusters of tasks that are not similar to each other. The proposed strategy efficiently utilizes the characteristics and properties of the MapReduce jobs in the given workload for constructing optimal job schedule. This resolves the problem of minimizing the makespan of workloads that additionally includes the workflow (DAGs) of mapreduce jobs.
Author Abinaya, B.
Raju, R.
Anuja, S.
Amudhavel, J.
Pavithra, M.
Author_xml – sequence: 1
  givenname: R.
  surname: Raju
  fullname: Raju, R.
  email: rajupdy@gmail.com
  organization: Bharathiyar Univ., Coimbatore, India
– sequence: 2
  givenname: J.
  surname: Amudhavel
  fullname: Amudhavel, J.
  email: info.amudhavel@gmail.com
  organization: Pondicherry Univ., Pondicherry, India
– sequence: 3
  givenname: M.
  surname: Pavithra
  fullname: Pavithra, M.
  email: pavismvec@gmail.com
  organization: SMVEC, Puducherry, India
– sequence: 4
  givenname: S.
  surname: Anuja
  fullname: Anuja, S.
  email: anuja.n1992@gmail.com
  organization: SMVEC, Puducherry, India
– sequence: 5
  givenname: B.
  surname: Abinaya
  fullname: Abinaya, B.
  email: abinayabskr25@gmail.com
  organization: SMVEC, Puducherry, India
BookMark eNotz71OwzAUQGEjwQClTwCDXyAh_ovjsYpCW6kICXWvbpxrsJo4keOAytMz0Olsn3QeyG0YAxLyzIqcscK87OttXTdNzgsm89JwLkt-Q9ZGV0xqY6SpuLgn7YZ-4RL9nLylDpY-0TT2GCEk-gbTB3aLReoiDPgzxjN1Y6SDD37wvz580gHOOE8QqA90d2mj72jdj0tHm_Dt4xgGDOmR3DnoZ1xfuyLH1-ZY77LD-3Zfbw6ZN0XKlLOFZi2UWncabSW4dqq0imkjmSuUAKV1WVmr29JYJRV0QmDVolAggaNYkad_1iPiaYp-gHg5XcfFH77hVB8
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/ICGCCEE.2014.6922462
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9781479949823
1479949825
EndPage 4
ExternalDocumentID 6922462
Genre orig-research
GroupedDBID 6IE
6IL
CBEJK
RIE
RIL
ID FETCH-LOGICAL-i90t-5fc071ba677d7ec8327f56c517941f053a57768cc7b69c545ad33e8be35a4a2e3
IEDL.DBID RIE
IngestDate Thu Jun 29 18:36:29 EDT 2023
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i90t-5fc071ba677d7ec8327f56c517941f053a57768cc7b69c545ad33e8be35a4a2e3
PageCount 4
ParticipantIDs ieee_primary_6922462
PublicationCentury 2000
PublicationDate 2014-March
PublicationDateYYYYMMDD 2014-03-01
PublicationDate_xml – month: 03
  year: 2014
  text: 2014-March
PublicationDecade 2010
PublicationTitle 2014 International Conference on Green Computing Communication and Electrical Engineering (ICGCCEE)
PublicationTitleAbbrev ICGCCEE
PublicationYear 2014
Publisher IEEE
Publisher_xml – name: IEEE
Score 1.8092036
Snippet Cloud Computing propounds a striking option for business to pay only for the resources that were consumed. The prime challenge is to increase the MapReduce...
SourceID ieee
SourceType Publisher
StartPage 1
SubjectTerms Algorithm design and analysis
Cloud computing
Clustering algorithms
DAG
Dynamic scheduling
Hadoop
Heuristic algorithms
Johnson algorithm
Makespan
MapReduce
MPT
Processor scheduling
Title A heuristic fault tolerant MapReduce framework for minimizing makespan in Hybrid Cloud Environment
URI https://ieeexplore.ieee.org/document/6922462
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFA9zePCksonf5ODRbv1Im-YopXOCjiFDdhv5eMHi1o7ZCvrXm7R1Q_DiJYSQD3gh770k7_d7CN1wBRAKz3eYiIVDmCscpk2hNBjdSDQjdea5l0c6mcTzOZt20O0WCwMAdfAZDGy1_stXhazsU9kwYpb-zCjcPUqjBqvVouE8lw0fkvskSVMbrkUGbddfOVNqkzE6_N9iR6i_w97h6daqHKMO5D0k7vArVA2pMta8Wpa4LJZg7EyJn_j62fKvAtY_gVbYeKLYkoassi8zCV7xNzCaI8dZjsefFqOFk2VRKZzuYG59NBuls2TstNkRnIy5pRNqabwDwSNKFQVpDibVYSQt4xbxtDlaPDRCiqWkImLS-ElcBQHEAoKQE-5DcIK6eZHDKcLCB6W9kNrBRGtj8wkzt2YvYhCHPPLPUM-KZ7Fu-C8WrWTO_26-QAd2B5o4rUvULTcVXKF9-VFm75vretO-AY5lm_E
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFA9jCnpS2cRvc_Bot7VNm-YopXPDbQwZsttI0hcsdu2YraB_vUlXNwQvXkII-YAX8t5L8n6_h9AdjwE8YTsWE4GwCOsJiyldxAq0biSKkSrz3MuITibBfM6mDXS_xcIAQBV8Bh1Trf7y41yW5qms6zNDf6YV7p7JnFWjtWo8nN1j3WH4GIZRZAK2SKfu_CtrSmU0-kf_W-4YtXfoOzzd2pUT1ICshcQDfoVyQ6uMFS_TAhd5CtrSFHjMV8-GgRWw-gm1wtoXxYY2ZJl86Unwkr-B1h0ZTjI8-DQoLRymeRnjaAd0a6NZP5qFA6vOj2AlrFdYnpLaPxDcpzSmIPXRpMrzpeHcIrbSh4t7VF8mpKTCZ1J7Sjx2XQgEuB4n3AH3FDWzPIMzhIUDsbI9agYTpbTVJ0zfm22fQeBx3zlHLSOexWrDgLGoJXPxd_MtOhjMxqPFaDh5ukSHZjc2UVtXqFmsS7hG-_KjSN7XN9UGfgPbn586
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2014+International+Conference+on+Green+Computing+Communication+and+Electrical+Engineering+%28ICGCCEE%29&rft.atitle=A+heuristic+fault+tolerant+MapReduce+framework+for+minimizing+makespan+in+Hybrid+Cloud+Environment&rft.au=Raju%2C+R.&rft.au=Amudhavel%2C+J.&rft.au=Pavithra%2C+M.&rft.au=Anuja%2C+S.&rft.date=2014-03-01&rft.pub=IEEE&rft.spage=1&rft.epage=4&rft_id=info:doi/10.1109%2FICGCCEE.2014.6922462&rft.externalDocID=6922462