A Distributed Low-Complexity Coding Solution for Large-Scale Distributed FFT

In distributed computing, a number of available helper nodes assist in completing a task for the master node. In such setups, the failure or straggling of even a single helper node can significantly increase the processing time. Therefore, coded distributed computing has been the subject of many rec...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on communications Vol. 68; no. 11; pp. 6617 - 6628
Main Authors: Yazdanialahabadi, Arash, Ardakani, Masoud
Format: Journal Article
Language:English
Published: New York IEEE 01.11.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Coding
Complexity
Complexity theory
Computer networks
Decoding
Design optimization
Design parameters
Discrete Fourier transforms
distributed coding
Distributed computing
Distributed processing
Encoding
fast Fourier transform
Nodes
Raptor code
Reliability
Task analysis
ISSN:0090-6778, 1558-0857
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In distributed computing, a number of available helper nodes assist in completing a task for the master node. In such setups, the failure or straggling of even a single helper node can significantly increase the processing time. Therefore, coded distributed computing has been the subject of many recent studies. A problem that arises in some setups is that the master's decoding complexity may exceed the complexity of self-computation, rending distributed computing useless. One such case is distributed large-scale FFT, where many helper nodes are required. In this work, we propose a novel distributed coded FFT, where the master's load is significantly lower than the existing work. The gain is obtained by (1) using a novel distributed FFT structure which allows for reliable distributed coding at the Shuffle stage, and (2) using Raptor codes which enjoy a linear complexity at the cost of a small number of extra helper nodes. Numerical results are provided to support the benefits of our proposed solution and to optimize design parameters.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0090-6778
1558-0857
DOI:10.1109/TCOMM.2020.3016648