Improving compressed matrix multiplication using control variate method

The seminal work by Pagh [1] proposed a matrix multiplication algorithm for real-valued squared matrices called Compressed Matrix Multiplication (CMM) having a sparse matrix output product. The algorithm is based on a popular sketching technique called Count-Sketch [2] and Fast Fourier Transform (FF...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Information processing letters Ročník 187; s. 106517
Hlavní autoři: Verma, Bhisham Dev, Dubey, Punit Pankaj, Pratap, Rameshwar, Thakur, Manoj
Médium: Journal Article
Jazyk:angličtina
Vydáno: Elsevier B.V 01.01.2025
Témata:
Control variate
Count-Sketch
Matrix multiplication
Sketching algorithms
Control variate
Sketching algorithms
Matrix multiplication
Count-Sketch
ISSN:0020-0190, 1872-6119
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:The seminal work by Pagh [1] proposed a matrix multiplication algorithm for real-valued squared matrices called Compressed Matrix Multiplication (CMM) having a sparse matrix output product. The algorithm is based on a popular sketching technique called Count-Sketch [2] and Fast Fourier Transform (FFT). For input square matrices A and B of order n and the product matrix AB with Frobenius norm ||AB||F, the algorithm offers an unbiased estimate for each entry, i.e., (AB)i,j of the product matrix AB with a variance bounded by ||AB||F2/b, where b is the compressed bucket size. Thus, the variance will eventually become high for a small bucket size. In this work, we address the high variance problem of CMM with the help of a simple and practical technique based on classical variance reduction methods in statistics. Our techniques rely on the Control Variate (CV) method. We suggest rigorous theoretical analysis for variance reduction and complement it via supporting empirical evidence. •This work proposes an improvement to the compressed matrix multiplication (CMM) algorithm.•Our proposal, CV-CMM, is based on the control variate (CV) method.•Our proposal provides more accurate matrix product estimates compared to CMM.
ISSN:0020-0190
1872-6119
DOI:10.1016/j.ipl.2024.106517