Improving compressed matrix multiplication using control variate method

The seminal work by Pagh [1] proposed a matrix multiplication algorithm for real-valued squared matrices called Compressed Matrix Multiplication (CMM) having a sparse matrix output product. The algorithm is based on a popular sketching technique called Count-Sketch [2] and Fast Fourier Transform (FF...

Full description

Saved in:
Bibliographic Details
Published in:Information processing letters Vol. 187; p. 106517
Main Authors: Verma, Bhisham Dev, Dubey, Punit Pankaj, Pratap, Rameshwar, Thakur, Manoj
Format: Journal Article
Language:English
Published: Elsevier B.V 01.01.2025
Subjects:
Control variate
Count-Sketch
Matrix multiplication
Sketching algorithms
Control variate
Sketching algorithms
Matrix multiplication
Count-Sketch
ISSN:0020-0190, 1872-6119
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The seminal work by Pagh [1] proposed a matrix multiplication algorithm for real-valued squared matrices called Compressed Matrix Multiplication (CMM) having a sparse matrix output product. The algorithm is based on a popular sketching technique called Count-Sketch [2] and Fast Fourier Transform (FFT). For input square matrices A and B of order n and the product matrix AB with Frobenius norm ||AB||F, the algorithm offers an unbiased estimate for each entry, i.e., (AB)i,j of the product matrix AB with a variance bounded by ||AB||F2/b, where b is the compressed bucket size. Thus, the variance will eventually become high for a small bucket size. In this work, we address the high variance problem of CMM with the help of a simple and practical technique based on classical variance reduction methods in statistics. Our techniques rely on the Control Variate (CV) method. We suggest rigorous theoretical analysis for variance reduction and complement it via supporting empirical evidence. •This work proposes an improvement to the compressed matrix multiplication (CMM) algorithm.•Our proposal, CV-CMM, is based on the control variate (CV) method.•Our proposal provides more accurate matrix product estimates compared to CMM.
ISSN:0020-0190
1872-6119
DOI:10.1016/j.ipl.2024.106517