Efficient gradient descent algorithm with anderson acceleration for separable nonlinear models

Separable nonlinear models are pervasively employed in diverse disciplines, such as system identification, signal analysis, electrical engineering, and machine learning. Identifying these models inherently poses a non-convex optimization challenge. While gradient descent (GD) is a commonly adopted m...

Full description

Saved in:
Bibliographic Details
Published in:Nonlinear dynamics Vol. 113; no. 10; pp. 11371 - 11387
Main Authors: Chen, Guang-Yong, Lin, Xin, Xue, Peng, Gan, Min
Format: Journal Article
Language:English
Published: Dordrecht Springer Netherlands 01.05.2025
Springer Nature B.V
Subjects:
Algorithms
Applications of Nonlinear Dynamics and Chaos Theory
Classical Mechanics
Control
Convergence
Convexity
Dynamical Systems
Hessian matrices
Machine learning
Newton methods
Optimization
Physics
Physics and Astronomy
Signal analysis
Statistical Physics and Dynamical Systems
System identification
Vibration
Anderson acceleration
Robust parameter estimation
Hierarchical identification algorithm
Separable nonlinear problem
ISSN:0924-090X, 1573-269X
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Separable nonlinear models are pervasively employed in diverse disciplines, such as system identification, signal analysis, electrical engineering, and machine learning. Identifying these models inherently poses a non-convex optimization challenge. While gradient descent (GD) is a commonly adopted method, it is often plagued by suboptimal convergence rates and is highly dependent on the appropriate choice of step size. To mitigate these issues, we introduce an augmented GD algorithm enhanced with Anderson acceleration (AA), and propose a Hierarchical GD with Anderson acceleration (H-AAGD) method for efficient identification of separable nonlinear models. This novel approach transcends the conventional step size constraints of GD algorithms and considers the coupling relationships between different parameters during the optimization process, thereby enhancing the efficiency of the solution-finding process. Unlike the Newton method, our algorithm obviates the need for computing the inverse of the Hessian matrix, simplifying the computational process. Additionally, we theoretically analyze the convergence and complexity of the algorithm and validate its effectiveness through a series of numerical experiments.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0924-090X
1573-269X
DOI:10.1007/s11071-024-10651-6