MapReduce Parallel Programming Model: A State-of-the-Art Survey

With the development of information technologies, we have entered the era of Big Data. Google’s MapReduce programming model and its open-source implementation in Apache Hadoop have become the dominant model for data-intensive processing because of its simplicity, scalability, and fault tolerance. Ho...

Full description

Saved in:
Bibliographic Details
Published in:International journal of parallel programming Vol. 44; no. 4; pp. 832 - 866
Main Authors: Li, Ren, Hu, Haibo, Li, Heng, Wu, Yunsong, Yang, Jianxi
Format: Journal Article
Language:English
Published: New York Springer US 01.08.2016
Springer Nature B.V
Subjects:
Algorithms
Analysis
Batch processing
Big Data
Cloud computing
Computer programming
Computer Science
Data processing
Design
Employment
Fault tolerance
Flexibility
Freeware
Information technology
Mathematical models
Open source software
Optimization
Parallel programming
Processor Architectures
Run time (computers)
Scalability
Science
Semantic web
Software Engineering/Programming and Operating Systems
Source code
Studies
Theory of Computation
Utility computing
Cloud computing
Big data
Scalability
Hadoop
MapReduce
ISSN:0885-7458, 1573-7640
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:With the development of information technologies, we have entered the era of Big Data. Google’s MapReduce programming model and its open-source implementation in Apache Hadoop have become the dominant model for data-intensive processing because of its simplicity, scalability, and fault tolerance. However, several inherent limitations, such as lack of efficient scheduling and iteration computing mechanisms, seriously affect the efficiency and flexibility of MapReduce. To date, various approaches have been proposed to extend MapReduce model and improve runtime efficiency for different scenarios. In this review, we assess MapReduce to help researchers better understand these novel optimizations that have been taken to address its limitations. We first present the basic idea underlying MapReduce paradigm and describe several widely used open-source runtime systems. And then we discuss the main shortcomings of original MapReduce. We also review these MapReduce optimization approaches that have recently been put forward, and categorize them according to the characteristics and capabilities. Finally, we conclude the paper and suggest several research works that should be carried out in the future.
Bibliography:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:0885-7458
1573-7640
DOI:10.1007/s10766-015-0395-0