A round robin with multiple feedback job scheduler in Hadoop

Hadoop is a distributed software platform for processing big data on a large cluster, which implements core mechanism of Google's GFS and MapReduce. The MapReduce job scheduling algorithm is one of the core technologies of Hadoop. The default job scheduler of Hadoop is FIFO, which will start th...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2014 IEEE International Conference on Progress in Informatics and Computing S. 471 - 475
Hauptverfasser: Yintian Wang, Ruonan Rao, Yinglin Wang
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 01.05.2014
Schlagworte:
Benchmark testing
Educational institutions
Hadoop
Job Schedule
MapReduce
Round robin
round-robin with multiple feedback
Throughput
Time factors
ISBN:9781479920334, 1479920339
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Hadoop is a distributed software platform for processing big data on a large cluster, which implements core mechanism of Google's GFS and MapReduce. The MapReduce job scheduling algorithm is one of the core technologies of Hadoop. The default job scheduler of Hadoop is FIFO, which will start the job in the order as it is submitted, and this causes the job to be started later when it is submitted later. This paper uses the round robin with a multiple feedback algorithm to solve this problem. With this scheduler, the job which is submitted late, will get quick response and be started without long delay. And the results of experiments on the Hadoop benchmark GridMix indicate that this algorithm can reduce the average response time by 10%-50%.
ISBN:9781479920334
1479920339
DOI:10.1109/PIC.2014.6972380