Improving compressed matrix multiplication using control variate method

The seminal work by Pagh [1] proposed a matrix multiplication algorithm for real-valued squared matrices called Compressed Matrix Multiplication (CMM) having a sparse matrix output product. The algorithm is based on a popular sketching technique called Count-Sketch [2] and Fast Fourier Transform (FF...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Information processing letters Ročník 187; s. 106517
Hlavní autori: Verma, Bhisham Dev, Dubey, Punit Pankaj, Pratap, Rameshwar, Thakur, Manoj
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Elsevier B.V 01.01.2025
Predmet:
Control variate
Count-Sketch
Matrix multiplication
Sketching algorithms
Control variate
Sketching algorithms
Matrix multiplication
Count-Sketch
ISSN:0020-0190, 1872-6119
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:The seminal work by Pagh [1] proposed a matrix multiplication algorithm for real-valued squared matrices called Compressed Matrix Multiplication (CMM) having a sparse matrix output product. The algorithm is based on a popular sketching technique called Count-Sketch [2] and Fast Fourier Transform (FFT). For input square matrices A and B of order n and the product matrix AB with Frobenius norm ||AB||F, the algorithm offers an unbiased estimate for each entry, i.e., (AB)i,j of the product matrix AB with a variance bounded by ||AB||F2/b, where b is the compressed bucket size. Thus, the variance will eventually become high for a small bucket size. In this work, we address the high variance problem of CMM with the help of a simple and practical technique based on classical variance reduction methods in statistics. Our techniques rely on the Control Variate (CV) method. We suggest rigorous theoretical analysis for variance reduction and complement it via supporting empirical evidence. •This work proposes an improvement to the compressed matrix multiplication (CMM) algorithm.•Our proposal, CV-CMM, is based on the control variate (CV) method.•Our proposal provides more accurate matrix product estimates compared to CMM.
ISSN:0020-0190
1872-6119
DOI:10.1016/j.ipl.2024.106517