A Distributed Low-Complexity Coding Solution for Large-Scale Distributed FFT

In distributed computing, a number of available helper nodes assist in completing a task for the master node. In such setups, the failure or straggling of even a single helper node can significantly increase the processing time. Therefore, coded distributed computing has been the subject of many rec...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on communications Jg. 68; H. 11; S. 6617 - 6628
Hauptverfasser: Yazdanialahabadi, Arash, Ardakani, Masoud
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York IEEE 01.11.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Coding
Complexity
Complexity theory
Computer networks
Decoding
Design optimization
Design parameters
Discrete Fourier transforms
distributed coding
Distributed computing
Distributed processing
Encoding
fast Fourier transform
Nodes
Raptor code
Reliability
Task analysis
ISSN:0090-6778, 1558-0857
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In distributed computing, a number of available helper nodes assist in completing a task for the master node. In such setups, the failure or straggling of even a single helper node can significantly increase the processing time. Therefore, coded distributed computing has been the subject of many recent studies. A problem that arises in some setups is that the master's decoding complexity may exceed the complexity of self-computation, rending distributed computing useless. One such case is distributed large-scale FFT, where many helper nodes are required. In this work, we propose a novel distributed coded FFT, where the master's load is significantly lower than the existing work. The gain is obtained by (1) using a novel distributed FFT structure which allows for reliable distributed coding at the Shuffle stage, and (2) using Raptor codes which enjoy a linear complexity at the cost of a small number of extra helper nodes. Numerical results are provided to support the benefits of our proposed solution and to optimize design parameters.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0090-6778
1558-0857
DOI:10.1109/TCOMM.2020.3016648