A round robin with multiple feedback job scheduler in Hadoop

Hadoop is a distributed software platform for processing big data on a large cluster, which implements core mechanism of Google's GFS and MapReduce. The MapReduce job scheduling algorithm is one of the core technologies of Hadoop. The default job scheduler of Hadoop is FIFO, which will start th...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2014 IEEE International Conference on Progress in Informatics and Computing s. 471 - 475
Hlavní autori: Yintian Wang, Ruonan Rao, Yinglin Wang
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 01.05.2014
Predmet:
Benchmark testing
Educational institutions
Hadoop
Job Schedule
MapReduce
Round robin
round-robin with multiple feedback
Throughput
Time factors
ISBN:9781479920334, 1479920339
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Hadoop is a distributed software platform for processing big data on a large cluster, which implements core mechanism of Google's GFS and MapReduce. The MapReduce job scheduling algorithm is one of the core technologies of Hadoop. The default job scheduler of Hadoop is FIFO, which will start the job in the order as it is submitted, and this causes the job to be started later when it is submitted later. This paper uses the round robin with a multiple feedback algorithm to solve this problem. With this scheduler, the job which is submitted late, will get quick response and be started without long delay. And the results of experiments on the Hadoop benchmark GridMix indicate that this algorithm can reduce the average response time by 10%-50%.
ISBN:9781479920334
1479920339
DOI:10.1109/PIC.2014.6972380