Automated tight Lyapunov analysis for first-order methods

Gespeichert in:
Bibliographische Detailangaben
Titel: Automated tight Lyapunov analysis for first-order methods
Autoren: Upadhyaya, Manu, Banert, Sebastian, Taylor, Adrien B., Giselsson, Pontus
Weitere Verfasser: Lund University, Faculty of Engineering, LTH, Departments at LTH, Department of Automatic Control, Lunds universitet, Lunds Tekniska Högskola, Institutioner vid LTH, Institutionen för reglerteknik, Originator, Lund University, Faculty of Engineering, LTH, LTH Profile areas, LTH Profile Area: AI and Digitalization, Lunds universitet, Lunds Tekniska Högskola, LTH profilområden, LTH profilområde: AI och digitalisering, Originator, Lund University, Profile areas and other strong research environments, Strategic research areas (SRA), ELLIIT: the Linköping-Lund initiative on IT and mobile communication, Lunds universitet, Profilområden och andra starka forskningsmiljöer, Strategiska forskningsområden (SFO), ELLIIT: the Linköping-Lund initiative on IT and mobile communication, Originator
Quelle: Mathematical Programming. 209(1-2):133-170
Schlagwörter: Engineering and Technology, Electrical Engineering, Electronic Engineering, Information Engineering, Control Engineering, Teknik, Elektroteknik och elektronik, Reglerteknik
Beschreibung: We present a methodology for establishing the existence of quadratic Lyapunov inequalities for a wide range of first-order methods used to solve convex optimization problems. In particular, we consider (i) classes of optimization problems of finite-sum form with (possibly strongly) convex and possibly smooth functional components, (ii) first-order methods that can be written as a linear system on state-space form in feedback interconnection with the subdifferentials of the functional components of the objective function, and (iii) quadratic Lyapunov inequalities that can be used to draw convergence conclusions. We present a necessary and sufficient condition for the existence of a quadratic Lyapunov inequality within a predefined class of Lyapunov inequalities, which amounts to solving a small-sized semidefinite program. We showcase our methodology on several first-order methods that fit the framework. Most notably, our methodology allows us to significantly extend the region of parameter choices that allow for duality gap convergence in the Chambolle–Pock method when the linear operator is the identity mapping.
Zugangs-URL: https://doi.org/10.1007/s10107-024-02061-8
Datenbank: SwePub
Beschreibung
Abstract:We present a methodology for establishing the existence of quadratic Lyapunov inequalities for a wide range of first-order methods used to solve convex optimization problems. In particular, we consider (i) classes of optimization problems of finite-sum form with (possibly strongly) convex and possibly smooth functional components, (ii) first-order methods that can be written as a linear system on state-space form in feedback interconnection with the subdifferentials of the functional components of the objective function, and (iii) quadratic Lyapunov inequalities that can be used to draw convergence conclusions. We present a necessary and sufficient condition for the existence of a quadratic Lyapunov inequality within a predefined class of Lyapunov inequalities, which amounts to solving a small-sized semidefinite program. We showcase our methodology on several first-order methods that fit the framework. Most notably, our methodology allows us to significantly extend the region of parameter choices that allow for duality gap convergence in the Chambolle–Pock method when the linear operator is the identity mapping.
ISSN:14364646
DOI:10.1007/s10107-024-02061-8