Improving compressed matrix multiplication using control variate method

The seminal work by Pagh [1] proposed a matrix multiplication algorithm for real-valued squared matrices called Compressed Matrix Multiplication (CMM) having a sparse matrix output product. The algorithm is based on a popular sketching technique called Count-Sketch [2] and Fast Fourier Transform (FF...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Information processing letters Jg. 187; S. 106517
Hauptverfasser: Verma, Bhisham Dev, Dubey, Punit Pankaj, Pratap, Rameshwar, Thakur, Manoj
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier B.V 01.01.2025
Schlagworte:
Control variate
Count-Sketch
Matrix multiplication
Sketching algorithms
Control variate
Sketching algorithms
Matrix multiplication
Count-Sketch
ISSN:0020-0190, 1872-6119
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The seminal work by Pagh [1] proposed a matrix multiplication algorithm for real-valued squared matrices called Compressed Matrix Multiplication (CMM) having a sparse matrix output product. The algorithm is based on a popular sketching technique called Count-Sketch [2] and Fast Fourier Transform (FFT). For input square matrices A and B of order n and the product matrix AB with Frobenius norm ||AB||F, the algorithm offers an unbiased estimate for each entry, i.e., (AB)i,j of the product matrix AB with a variance bounded by ||AB||F2/b, where b is the compressed bucket size. Thus, the variance will eventually become high for a small bucket size. In this work, we address the high variance problem of CMM with the help of a simple and practical technique based on classical variance reduction methods in statistics. Our techniques rely on the Control Variate (CV) method. We suggest rigorous theoretical analysis for variance reduction and complement it via supporting empirical evidence. •This work proposes an improvement to the compressed matrix multiplication (CMM) algorithm.•Our proposal, CV-CMM, is based on the control variate (CV) method.•Our proposal provides more accurate matrix product estimates compared to CMM.
ISSN:0020-0190
1872-6119
DOI:10.1016/j.ipl.2024.106517