A round robin with multiple feedback job scheduler in Hadoop

Hadoop is a distributed software platform for processing big data on a large cluster, which implements core mechanism of Google's GFS and MapReduce. The MapReduce job scheduling algorithm is one of the core technologies of Hadoop. The default job scheduler of Hadoop is FIFO, which will start th...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2014 IEEE International Conference on Progress in Informatics and Computing s. 471 - 475
Hlavní autoři: Yintian Wang, Ruonan Rao, Yinglin Wang
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.05.2014
Témata:
Benchmark testing
Educational institutions
Hadoop
Job Schedule
MapReduce
Round robin
round-robin with multiple feedback
Throughput
Time factors
ISBN:9781479920334, 1479920339
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Hadoop is a distributed software platform for processing big data on a large cluster, which implements core mechanism of Google's GFS and MapReduce. The MapReduce job scheduling algorithm is one of the core technologies of Hadoop. The default job scheduler of Hadoop is FIFO, which will start the job in the order as it is submitted, and this causes the job to be started later when it is submitted later. This paper uses the round robin with a multiple feedback algorithm to solve this problem. With this scheduler, the job which is submitted late, will get quick response and be started without long delay. And the results of experiments on the Hadoop benchmark GridMix indicate that this algorithm can reduce the average response time by 10%-50%.
ISBN:9781479920334
1479920339
DOI:10.1109/PIC.2014.6972380