Gradient-based algorithms for multi-objective bi-level optimization

Multi-objective bi-level optimization (MOBLO) addresses nested multi-objective optimization problems common in a range of applications. However, its multi-objective and hierarchical bi-level nature makes it notably complex. Gradient-based MOBLO algorithms have recently grown in popularity, as they e...

Full description

Saved in:
Bibliographic Details
Published in:Science China. Mathematics Vol. 67; no. 6; pp. 1419 - 1438
Main Authors: Yang, Xinmin, Yao, Wei, Yin, Haian, Zeng, Shangzhi, Zhang, Jin
Format: Journal Article
Language:English
Published: Beijing Science China Press 01.06.2024
Springer Nature B.V
Subjects:
Algorithms
Applications of Mathematics
Complexity
Machine learning
Mathematics
Mathematics and Statistics
Multiple objective analysis
Neural networks
Optimization
Pareto optimization
90C30
convergence analysis
bi-level optimization
Pareto stationary
90C29
90C26
multi-objective
learning to optimize
ISSN:1674-7283, 1869-1862
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Multi-objective bi-level optimization (MOBLO) addresses nested multi-objective optimization problems common in a range of applications. However, its multi-objective and hierarchical bi-level nature makes it notably complex. Gradient-based MOBLO algorithms have recently grown in popularity, as they effectively solve crucial machine learning problems like meta-learning, neural architecture search, and reinforcement learning. Unfortunately, these algorithms depend on solving a sequence of approximation subproblems with high accuracy, resulting in adverse time and memory complexity that lowers their numerical efficiency. To address this issue, we propose a gradient-based algorithm for MOBLO, called gMOBA, which has fewer hyperparameters to tune, making it both simple and efficient. Additionally, we demonstrate the theoretical validity by accomplishing the desirable Pareto stationarity. Numerical experiments confirm the practical efficiency of the proposed method and verify the theoretical results. To accelerate the convergence of gMOBA, we introduce a beneficial L2O (learning to optimize) neural network (called L2O-gMOBA) implemented as the initialization phase of our gMOBA algorithm. Comparative results of numerical experiments are presented to illustrate the performance of L2O-gMOBA.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1674-7283
1869-1862
DOI:10.1007/s11425-023-2302-9