Freyr ^++: Harvesting Idle Resources in Serverless Computing via Deep Reinforcement Learning

Serverless computing has revolutionized online service development and deployment with ease-to-use operations, auto-scaling, fine-grained resource allocation, and pay-as-you-go pricing. However, a gap remains in configuring serverless functions-the actual resource consumption may vary due to functio...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on parallel and distributed systems Ročník 35; číslo 11; s. 2254 - 2269
Hlavní autoři: Yu, Hanfei, Wang, Hao, Li, Jian, Yuan, Xu, Park, Seung-Jong
Médium: Journal Article
Jazyk:angličtina
Vydáno: IEEE 01.11.2024
Témata:
Attention
Deep reinforcement learning
Dynamic scheduling
Heuristic algorithms
incremental learning
Memory management
Nearest neighbor methods
reinforcement learning
resource harvesting
Serverless computing
Training
ISSN:1045-9219, 1558-2183
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Serverless computing has revolutionized online service development and deployment with ease-to-use operations, auto-scaling, fine-grained resource allocation, and pay-as-you-go pricing. However, a gap remains in configuring serverless functions-the actual resource consumption may vary due to function types, dependencies, and input data sizes, thus mismatching the static resource configuration by users. Dynamic resource consumption against static configuration may lead to either poor function execution performance or low utilization. This paper proposes Freyr <inline-formula><tex-math notation="LaTeX">^+</tex-math> <mml:math><mml:msup><mml:mrow/><mml:mo>+</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="yu-ieq3-3462294.gif"/> </inline-formula>, a novel resource manager (RM) that dynamically harvests idle resources from over-provisioned functions to accelerate under-provisioned functions for serverless platforms. Freyr <inline-formula><tex-math notation="LaTeX">^+</tex-math> <mml:math><mml:msup><mml:mrow/><mml:mo>+</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="yu-ieq4-3462294.gif"/> </inline-formula> monitors each function's resource utilization in real-time and detects the mismatches between user configuration and actual resource consumption. We design deep reinforcement learning (DRL) algorithms with attention-enhanced embedding, incremental learning, and safeguard mechanism for Freyr <inline-formula><tex-math notation="LaTeX">^+</tex-math> <mml:math><mml:msup><mml:mrow/><mml:mo>+</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="yu-ieq5-3462294.gif"/> </inline-formula> to harvest idle resources safely and accelerate functions efficiently. We have implemented and deployed a Freyr <inline-formula><tex-math notation="LaTeX">^+</tex-math> <mml:math><mml:msup><mml:mrow/><mml:mo>+</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="yu-ieq6-3462294.gif"/> </inline-formula> prototype in a 13-node Apache OpenWhisk cluster using AWS EC2. Freyr <inline-formula><tex-math notation="LaTeX">^+</tex-math> <mml:math><mml:msup><mml:mrow/><mml:mo>+</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="yu-ieq7-3462294.gif"/> </inline-formula> is evaluated on both large-scale simulation and real-world testbed. Experimental results show that Freyr <inline-formula><tex-math notation="LaTeX">^+</tex-math> <mml:math><mml:msup><mml:mrow/><mml:mo>+</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="yu-ieq8-3462294.gif"/> </inline-formula> harvests 38% of function invocations' idle resources and accelerates 39% of invocations using harvested resources. Freyr <inline-formula><tex-math notation="LaTeX">^+</tex-math> <mml:math><mml:msup><mml:mrow/><mml:mo>+</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="yu-ieq9-3462294.gif"/> </inline-formula> reduces the 99th-percentile function response latency by 26% compared to the baseline RMs.
ISSN:1045-9219
1558-2183
DOI:10.1109/TPDS.2024.3462294