A fast randomized algorithm for overdetermined linear least-squares regression - Rokhlin, Vladimir; Tygert, Mark (Proceedings of the National Academy of Sciences - PNAS, 2008)
We introduce a randomized algorithm for overdetermined linear least-squares regression. Given an arbitrary full-rank m x n matrix A with m >/= n, any m x 1 vector b, and any positive real number epsilon, the procedure computes an n x 1 vector x such that x minimizes the Euclidean norm ||Ax - b || to relative precision epsilon. The algorithm typically requires ((log(n)+log(1/epsilon))mn+n(3)) floating-point operations. This cost is less than the (mn(2)) required by the classical schemes based on QR-decompositions or bidiagonalization. We present several numerical examples illustrating the performance of the algorithm.