An optimal control framework for adaptive neural ODEs

In recent years, the notion of neural ODEs has connected deep learning with the field of ODEs and optimal control. In this setting, neural networks are defined as the mapping induced by the corresponding time-discretization scheme of a given ODE. The learning task consists in finding the ODE paramet...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Advances in computational mathematics Ročník 50; číslo 3; s. 52
Hlavní autori: Aghili, Joubine, Mula, Olga
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York Springer US 01.06.2024
Springer Nature B.V
Predmet:
Accuracy
Adaptive sampling
Algorithms
Approximation
Cognitive tasks
Computational mathematics
Computational Mathematics and Numerical Analysis
Computational Science and Engineering
Deep learning
Discretization
Dynamical systems
Mathematical and Computational Biology
Mathematical Modeling and Industrial Mathematics
Mathematics
Mathematics and Statistics
Neural networks
Optimal control
Optimization
Sampling error
Visualization
Deep learning
Adaptivity
68T05
Optimal control
65K10
68Q32
65L10
49J15
Neural ODE networks
ISSN:1019-7168, 1572-9044
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:In recent years, the notion of neural ODEs has connected deep learning with the field of ODEs and optimal control. In this setting, neural networks are defined as the mapping induced by the corresponding time-discretization scheme of a given ODE. The learning task consists in finding the ODE parameters as the optimal values of a sampled loss minimization problem. In the limit of infinite time steps, and data samples, we obtain a notion of continuous formulation of the problem. The practical implementation involves two discretization errors: a sampling error and a time-discretization error. In this work, we develop a general optimal control framework to analyze the interplay between the above two errors. We prove that to approximate the solution of the fully continuous problem at a certain accuracy, we not only need a minimal number of training samples, but also need to solve the control problem on the sampled loss function with some minimal accuracy. The theoretical analysis allows us to develop rigorous adaptive schemes in time and sampling, and gives rise to a notion of adaptive neural ODEs. The performance of the approach is illustrated in several numerical examples.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1019-7168
1572-9044
DOI:10.1007/s10444-024-10149-0