Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning

Modern problems in AI or in numerical analysis require nonsmooth approaches with a flexible calculus. We introduce generalized derivatives called conservative fields for which we develop a calculus and provide representation formulas. Functions having a conservative field are called path differentia...

Full description

Saved in:

Bibliographic Details
Published in:	Mathematical programming Vol. 188; no. 1; pp. 19 - 51
Main Authors:	Bolte, Jérôme, Pauwels, Edouard
Format:	Journal Article
Language:	English
Published:	Berlin/Heidelberg Springer Berlin Heidelberg 01.07.2021 Springer Nature B.V
Subjects:	Algorithms Back propagation Calculus Calculus of Variations and Optimal Control; Optimization Combinatorics Continuity (mathematics) Deep learning Differentiation Full Length Paper Machine learning Mathematical analysis Mathematical and Computational Physics Mathematical Methods in Physics Mathematics Mathematics and Statistics Mathematics of Computing Numerical Analysis Theoretical o-Minimal structures 62M45 Neural nets and related approaches to inference from stochastic processes 90C06 Large-scale problems 68T05 Learning and adaptive systems in artificial intelligence Stochastic gradient Automatic differentiation Deep learning Backpropagation algorithm Clarke subdifferential 49M27 Decomposition methods First order methods Nonsmooth stochastic optimization Definable sets 65K10 Numerical optimization and variational techniques 49J53 Set-valued and variational analysis
ISSN:	0025-5610, 1436-4646
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Be the first to leave a comment!