Efficient gradient descent algorithm with anderson acceleration for separable nonlinear models

Separable nonlinear models are pervasively employed in diverse disciplines, such as system identification, signal analysis, electrical engineering, and machine learning. Identifying these models inherently poses a non-convex optimization challenge. While gradient descent (GD) is a commonly adopted m...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Nonlinear dynamics Ročník 113; číslo 10; s. 11371 - 11387
Hlavní autoři: Chen, Guang-Yong, Lin, Xin, Xue, Peng, Gan, Min
Médium: Journal Article
Jazyk:angličtina
Vydáno: Dordrecht Springer Netherlands 01.05.2025
Springer Nature B.V
Témata:
Algorithms
Applications of Nonlinear Dynamics and Chaos Theory
Classical Mechanics
Control
Convergence
Convexity
Dynamical Systems
Hessian matrices
Machine learning
Newton methods
Optimization
Physics
Physics and Astronomy
Signal analysis
Statistical Physics and Dynamical Systems
System identification
Vibration
Anderson acceleration
Robust parameter estimation
Hierarchical identification algorithm
Separable nonlinear problem
ISSN:0924-090X, 1573-269X
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Separable nonlinear models are pervasively employed in diverse disciplines, such as system identification, signal analysis, electrical engineering, and machine learning. Identifying these models inherently poses a non-convex optimization challenge. While gradient descent (GD) is a commonly adopted method, it is often plagued by suboptimal convergence rates and is highly dependent on the appropriate choice of step size. To mitigate these issues, we introduce an augmented GD algorithm enhanced with Anderson acceleration (AA), and propose a Hierarchical GD with Anderson acceleration (H-AAGD) method for efficient identification of separable nonlinear models. This novel approach transcends the conventional step size constraints of GD algorithms and considers the coupling relationships between different parameters during the optimization process, thereby enhancing the efficiency of the solution-finding process. Unlike the Newton method, our algorithm obviates the need for computing the inverse of the Hessian matrix, simplifying the computational process. Additionally, we theoretically analyze the convergence and complexity of the algorithm and validate its effectiveness through a series of numerical experiments.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0924-090X
1573-269X
DOI:10.1007/s11071-024-10651-6