Energy efficient resource controller for Apache Storm

Summary Apache Storm is a distributed processing engine that can reliably process unbounded streams of data for real‐time applications. While recent research activities mostly focused on devising a resource allocation and task scheduling algorithm to satisfy high performance or low latency requireme...

Full description

Saved in:
Bibliographic Details
Published in:Concurrency and computation Vol. 35; no. 17
Main Authors: HoseinyFarahabady, MohammadReza, Taheri, Javid, Zomaya, Albert Y., Tari, Zahir
Format: Journal Article
Language:English
Published: Hoboken Wiley Subscription Services, Inc 01.08.2023
Subjects:
Algorithms
Computer Science
Controllers
data stream processing engines
Datavetenskap
Distributed processing
Energy consumption
energy-aware resource allocation algorithm
performance evaluation of computer systems
Resource allocation
Resource scheduling
Task scheduling
Throttling
ISSN:1532-0626, 1532-0634, 1532-0634
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Summary Apache Storm is a distributed processing engine that can reliably process unbounded streams of data for real‐time applications. While recent research activities mostly focused on devising a resource allocation and task scheduling algorithm to satisfy high performance or low latency requirements of Storm applications across a distributed and multi‐core system, finding a solution that can optimize the energy consumption of running applications remains an important research question to be further explored. In this article, we present a controlling strategy for CPU throttling that continuously optimize the level of consumed energy of a Storm platform by adjusting the voltage and frequency of the CPU cores while running the assigned tasks under latency constraints defined by the end‐users. The experimental results running over a Storm cluster with 4 physical nodes (total 24 cores) validates the effectiveness of proposed solution when running multiple compute‐intensive operations. In particular, the proposed controller can keep the latency of analytic tasks, in terms of 99th latency percentile, within the quality of service requirement specified by the end‐user while reducing the total energy consumption by 18% on average across the entire Storm platform.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1532-0626
1532-0634
1532-0634
DOI:10.1002/cpe.6799