Transition Waste Optimization for Coded Elastic Computing

Distributed computing, in which a resource-intensive task is divided into subtasks and distributed among different machines, plays a key role in solving large-scale problems. Coded computing is a recently emerging paradigm where redundancy for distributed computing is introduced to alleviate the imp...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on information theory Ročník 69; číslo 7; s. 4442 - 4465
Hlavní autoři: Dau, Son Hoang, Gabrys, Ryan, Huang, Yu-Chih, Feng, Chen, Luu, Quang-Hung, Alzahrani, Eidah J., Tari, Zahir
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.07.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Cloud computing
coded computing
Completion time
Computer networks
Costs
Distributed computing
Distributed processing
erasure coding
finite geometry
Industries
Optimization
Redundancy
resource allocation
Resource management
Task analysis
Toy manufacturing industry
Virtual environments
Virtual machining
ISSN:0018-9448, 1557-9654
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Distributed computing, in which a resource-intensive task is divided into subtasks and distributed among different machines, plays a key role in solving large-scale problems. Coded computing is a recently emerging paradigm where redundancy for distributed computing is introduced to alleviate the impact of slow machines (stragglers) on the completion time. We investigate coded computing solutions over elastic resources, where the set of available machines may change in the middle of the computation. This is motivated by recently available services in the cloud computing industry (e.g., EC2 Spot, Azure Batch) where low-priority virtual machines are offered at a fraction of the price of the on- demand instances but can be preempted on short notice. Our contributions are three-fold. We first introduce a new concept called transition waste that quantifies the number of tasks existing machines must abandon or take over when a machine joins/leaves. We then develop an efficient method to minimize the transition waste for the cyclic task allocation scheme recently proposed in the literature (Yang et al. ISIT'19). Finally, we establish a novel solution based on finite geometry achieving zero transition wastes given that the number of active machines varies within a fixed range.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2023.3247860