DMR: A Deterministic MapReduce for Multicore Systems

MapReduce has been shown promising to harness the multicore platform. Existing MapReduce libraries on multicore are written with shared-memory Pthreads, which introduce pervasive nondeterminism and might produce nondeterministic results if user-provided map or reduce functions are sensitive to the i...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:International journal of parallel programming Ročník 45; číslo 1; s. 128 - 141
Hlavní autoři: Zhang, Yu, Cao, Huifang
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.02.2017
Springer Nature B.V
Témata:
Analysis
Central processing units
Computer programming
Computer Science
CPUs
Design of experiments
Harnesses
Libraries
Mathematical models
Parallel processing
Platforms
Processor Architectures
Scheduling
Software Engineering/Programming and Operating Systems
Studies
Task scheduling
Tasks
Theory of Computation
Variables
Workload
Workloads
Multicore
Deterministic parallelism
MapReduce
ISSN:0885-7458, 1573-7640
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:MapReduce has been shown promising to harness the multicore platform. Existing MapReduce libraries on multicore are written with shared-memory Pthreads, which introduce pervasive nondeterminism and might produce nondeterministic results if user-provided map or reduce functions are sensitive to the input order. We propose DMR, a deterministic MapReduce library, to ensure deterministic program behaviors no matter whether map/reduce function is sensitive to the input order. DMR adopts a round-robin scheduling of map tasks and a partitioned scheduling of reduce tasks to ensure deterministic scheduling. DMR is written with a deterministic message passing multithreaded model (DetMP) to provide Phoenix-like API, thus Phoenix workloads can be built and run on DMR with no or little change. Evaluation results by testing seven Phoenix workloads show that DMR only runs worse than Phoenix on an iterative MapReduce application kmeans, outperforms Phoenix between 1.42X and 3.33X faster on pca and word_count, and scales better than Phoenix on 3 of the rest 4 workloads.
Bibliografie:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:0885-7458
1573-7640
DOI:10.1007/s10766-015-0390-5