Generalized Fractional Repetition Codes for Binary Coded Computations

This paper addresses the gradient coding and coded matrix multiplication problems in distributed optimization and coded computing. We present a computationally efficient coding method which overcomes the drawbacks of the Fractional Repetition Coding gradient coding method proposed by Tandon et al.,...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on information theory Jg. 71; H. 3; S. 2170 - 2194
Hauptverfasser: Charalambides, Neophytos, Mahdavifar, Hessam, Hero, Alfred O.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: IEEE 01.03.2025
Schlagworte:
Accuracy
binary erasure codes
Codes
Computer science
Decoding
Distributed gradient descent
distributed matrix multiplication
Encoding
fractional repetition codes
numerical accuracy
Numerical stability
Optimization
Resource management
Servers
straggler mitigation
Vectors
ISSN:0018-9448, 1557-9654
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This paper addresses the gradient coding and coded matrix multiplication problems in distributed optimization and coded computing. We present a computationally efficient coding method which overcomes the drawbacks of the Fractional Repetition Coding gradient coding method proposed by Tandon et al., and can also be leveraged by coded computing networks whose servers are of heterogeneous nature. Specifically, we propose a construction for fractional repetition gradient coding; while ensuring that the generator matrix remains close to perfectly balanced for any set of coding parameters, as well as a low complexity decoding step. The proposed binary encoding avoids operations over the real and complex numbers which inherently introduce numerical and rounding errors, thereby enabling accurate distributed encodings of the partial gradients. We then make connections between gradient coding and coded matrix multiplication. Specifically, we show that any gradient coding scheme can be extended to coded matrix multiplication. Furthermore, we show how the proposed binary gradient coding scheme can be used to construct two different coded matrix multiplication schemes, each achieving different trade-offs.
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2025.3529680