ANDERSON ACCELERATION FOR FIXED-POINT ITERATIONS

This paper concerns an acceleration method for fixed-point iterations that originated in work of D. G. Anderson [J. Assoc. Comput. Mach., 12 (1965), pp. 547–560], which we accordingly call Anderson acceleration here. This method has enjoyed considerable success and wide usage in electronic structure...

Full description

Saved in:
Bibliographic Details
Published in:SIAM journal on numerical analysis Vol. 49; no. 3/4; pp. 1715 - 1735
Main Authors: WALKER, HOMER F., NI, PENG
Format: Journal Article
Language:English
Published: Philadelphia, PA Society for Industrial and Applied Mathematics 01.01.2011
Subjects:
Algebra
Algorithms
Applied mathematics
Cost efficiency
Exact sciences and technology
Experiments
Global analysis, analysis on manifolds
Iterative solutions
Jacobians
Linear algebra
Linear and multilinear algebra, matrix theory
Linear systems
Mathematical extrapolation
Mathematical vectors
Mathematics
Matrices
Methods
Numerical analysis
Numerical analysis in abstract spaces
Numerical analysis. Scientific computation
Numerical methods in probability and statistics
Sciences and techniques of general use
Secant function
Topology. Manifolds and cell complexes. Global analysis and analysis on manifolds
Truncation
Orthogonal matrix
iterative methods
Arnoldi method
Decomposition method
Iterative method
Iteration
Numerical method
Stochastic method
Partial differential equation
Truncation
Fix point
65F10
Multigrid
Least squares method
mixture densities
65H10
acceleration methods
Arnoldi (full orthogonalization) method
expectation-maximization algorithm
Quasi Newton method
Numerical linear algebra
Initial value problem
fixed-point iterations
Algorithm
nonnegative matrix factorization
GMRES method
Numerical analysis
Boundary value problem
Linear system
Linear algebra
Abstract space
alternating least-squares
generalized minimal residual method
Domain decomposition
ISSN:0036-1429, 1095-7170
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper concerns an acceleration method for fixed-point iterations that originated in work of D. G. Anderson [J. Assoc. Comput. Mach., 12 (1965), pp. 547–560], which we accordingly call Anderson acceleration here. This method has enjoyed considerable success and wide usage in electronic structure computations, where it is known as Anderson mixing; however, it seems to have been untried or underexploited in many other important applications. Moreover, while other acceleration methods have been extensively studied by the mathematics and numerical analysis communities, this method has received relatively little attention from these communities over the years. A recent paper by H. Fang and Y. Saad [Numer. Linear Algebra Appl., 16 (2009), pp. 197–221] has clarified a remarkable relationship of Anderson acceleration to quasi-Newton (secant updating) methods and extended it to define a broader Anderson family of acceleration methods. In this paper, our goals are to shed additional light on Anderson acceleration and to draw further attention to its usefulness as a general tool. We first show that, on linear problems, Anderson acceleration without truncation is "essentially equivalent" in a certain sense to the generalized minimal residual (GMRES) method. We also show that the Type 1 variant in the Fang—Saad Anderson family is similarly essentially equivalent to the Arnoldi (full orthogonalization) method. We then discuss practical considerations for implementing Anderson acceleration and illustrate its performance through numerical experiments involving a variety of applications.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
content type line 14
ISSN:0036-1429
1095-7170
DOI:10.1137/10078356x