Dynamic energy-efficient scheduling for streaming applications in storm

With the rapid development of information technology, the data generated by the Internet has exploded in recent years. The proliferation of data has brought about a huge increase in the energy consumption for data processing especially in real-time processing framework for big data. In this study, t...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Computing Ročník 104; číslo 2; s. 413 - 432
Hlavní autori: Li, Hongjian, Dai, Hongxi, Liu, Zengyan, Fu, Hao, Zou, Yang
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Vienna Springer Vienna 01.02.2022
Springer Nature B.V
Predmet:
Algorithms
Artificial Intelligence
Big Data
Clusters
Computer Appl. in Administrative Data Processing
Computer Communication Networks
Computer Science
Data processing
Energy consumption
Information Systems Applications (incl.Internet)
Nodes
Regular Paper
Scheduling
Software Engineering
Slot usage
Streaming application
Load balancing
Apache storm
Energy-efficient scheduling
68Q85
ISSN:0010-485X, 1436-5057
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:With the rapid development of information technology, the data generated by the Internet has exploded in recent years. The proliferation of data has brought about a huge increase in the energy consumption for data processing especially in real-time processing framework for big data. In this study, two energy-efficient scheduling algorithms are proposed to reduce energy consumption for streaming applications in Storm. First, an energy consumption model is designed for Storm framework. Then this model is introduced into Storm by an energy consumption monitoring module. For proposed algorithm 1, the energy consumption of the processing tasks is minimized by integrating the tasks into the low energy consumption nodes. For proposed algorithm 2, load balance and energy consumption of Storm cluster are traded off and optimized by sorting the Slot utilization of low energy consumption nodes in the cluster and assigning tasks priority to the low Slot utilization nodes. Test on Hibench workload, the proposed algorithms reduce the total energy consumption of Storm cluster up to 32% compared with the traditional scheduling algorithms. It shows that the proposed scheduling algorithms can effectively reduce the total energy consumption of the Storm cluster while satisfying the deadline constrains.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0010-485X
1436-5057
DOI:10.1007/s00607-021-00961-7