An optimal control framework for adaptive neural ODEs

In recent years, the notion of neural ODEs has connected deep learning with the field of ODEs and optimal control. In this setting, neural networks are defined as the mapping induced by the corresponding time-discretization scheme of a given ODE. The learning task consists in finding the ODE paramet...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Advances in computational mathematics Jg. 50; H. 3; S. 52
Hauptverfasser: Aghili, Joubine, Mula, Olga
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.06.2024
Springer Nature B.V
Schlagworte:
Accuracy
Adaptive sampling
Algorithms
Approximation
Cognitive tasks
Computational mathematics
Computational Mathematics and Numerical Analysis
Computational Science and Engineering
Deep learning
Discretization
Dynamical systems
Mathematical and Computational Biology
Mathematical Modeling and Industrial Mathematics
Mathematics
Mathematics and Statistics
Neural networks
Optimal control
Optimization
Sampling error
Visualization
Deep learning
Adaptivity
68T05
Optimal control
65K10
68Q32
65L10
49J15
Neural ODE networks
ISSN:1019-7168, 1572-9044
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In recent years, the notion of neural ODEs has connected deep learning with the field of ODEs and optimal control. In this setting, neural networks are defined as the mapping induced by the corresponding time-discretization scheme of a given ODE. The learning task consists in finding the ODE parameters as the optimal values of a sampled loss minimization problem. In the limit of infinite time steps, and data samples, we obtain a notion of continuous formulation of the problem. The practical implementation involves two discretization errors: a sampling error and a time-discretization error. In this work, we develop a general optimal control framework to analyze the interplay between the above two errors. We prove that to approximate the solution of the fully continuous problem at a certain accuracy, we not only need a minimal number of training samples, but also need to solve the control problem on the sampled loss function with some minimal accuracy. The theoretical analysis allows us to develop rigorous adaptive schemes in time and sampling, and gives rise to a notion of adaptive neural ODEs. The performance of the approach is illustrated in several numerical examples.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1019-7168
1572-9044
DOI:10.1007/s10444-024-10149-0