DMR: A Deterministic MapReduce for Multicore Systems

MapReduce has been shown promising to harness the multicore platform. Existing MapReduce libraries on multicore are written with shared-memory Pthreads, which introduce pervasive nondeterminism and might produce nondeterministic results if user-provided map or reduce functions are sensitive to the i...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:International journal of parallel programming Jg. 45; H. 1; S. 128 - 141
Hauptverfasser: Zhang, Yu, Cao, Huifang
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.02.2017
Springer Nature B.V
Schlagworte:
Analysis
Central processing units
Computer programming
Computer Science
CPUs
Design of experiments
Harnesses
Libraries
Mathematical models
Parallel processing
Platforms
Processor Architectures
Scheduling
Software Engineering/Programming and Operating Systems
Studies
Task scheduling
Tasks
Theory of Computation
Variables
Workload
Workloads
Multicore
Deterministic parallelism
MapReduce
ISSN:0885-7458, 1573-7640
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:MapReduce has been shown promising to harness the multicore platform. Existing MapReduce libraries on multicore are written with shared-memory Pthreads, which introduce pervasive nondeterminism and might produce nondeterministic results if user-provided map or reduce functions are sensitive to the input order. We propose DMR, a deterministic MapReduce library, to ensure deterministic program behaviors no matter whether map/reduce function is sensitive to the input order. DMR adopts a round-robin scheduling of map tasks and a partitioned scheduling of reduce tasks to ensure deterministic scheduling. DMR is written with a deterministic message passing multithreaded model (DetMP) to provide Phoenix-like API, thus Phoenix workloads can be built and run on DMR with no or little change. Evaluation results by testing seven Phoenix workloads show that DMR only runs worse than Phoenix on an iterative MapReduce application kmeans, outperforms Phoenix between 1.42X and 3.33X faster on pca and word_count, and scales better than Phoenix on 3 of the rest 4 workloads.
Bibliographie:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:0885-7458
1573-7640
DOI:10.1007/s10766-015-0390-5