Scalable subspace methods for derivative-free nonlinear least-squares optimization

We introduce a general framework for large-scale model-based derivative-free optimization based on iterative minimization within random subspaces. We present a probabilistic worst-case complexity analysis for our method, where in particular we prove high-probability bounds on the number of iteration...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	Mathematical programming Ročník 199; číslo 1-2; s. 461 - 524
Hlavní autoři:	Cartis, Coralia, Roberts, Lindon
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	Berlin/Heidelberg Springer Berlin Heidelberg 01.05.2023 Springer
Témata:	Algebra Calculus of Variations and Optimal Control; Optimization Combinatorics Full Length Paper Mathematical and Computational Physics Mathematical Methods in Physics Mathematics Mathematics and Statistics Mathematics of Computing Methods Numerical Analysis Theoretical 65K05 90C30 Worst case complexity 90C56 Large-scale optimization Nonlinear least-squares Derivative-free optimization
ISSN:	0025-5610, 1436-4646
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Popis
Shrnutí:	We introduce a general framework for large-scale model-based derivative-free optimization based on iterative minimization within random subspaces. We present a probabilistic worst-case complexity analysis for our method, where in particular we prove high-probability bounds on the number of iterations before a given optimality is achieved. This framework is specialized to nonlinear least-squares problems, with a model-based framework based on the Gauss–Newton method. This method achieves scalability by constructing local linear interpolation models to approximate the Jacobian, and computes new steps at each iteration in a subspace with user-determined dimension. We then describe a practical implementation of this framework, which we call DFBGN. We outline efficient techniques for selecting the interpolation points and search subspace, yielding an implementation that has a low per-iteration linear algebra cost (linear in the problem dimension) while also achieving fast objective decrease as measured by evaluations. Extensive numerical results demonstrate that DFBGN has improved scalability, yielding strong performance on large-scale nonlinear least-squares problems.
ISSN:	0025-5610 1436-4646
DOI:	10.1007/s10107-022-01836-1