A Universal Approximation Result for Difference of Log-Sum-Exp Neural Networks

We show that a neural network whose output is obtained as the difference of the outputs of two feedforward networks with exponential activation function in the hidden layer and logarithmic activation function in the output node, referred to as log-sum-exp (LSE) network, is a smooth universal approxi...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE transaction on neural networks and learning systems Ročník 31; číslo 12; s. 5603 - 5612
Hlavní autori: Calafiore, Giuseppe C., Gaubert, Stephane, Possieri, Corrado
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: United States IEEE 01.12.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Algorithms
Artificial neural networks
Computer Science
Continuity (mathematics)
Convex functions
Data-driven optimization
Design
Design optimization
Diabetes mellitus
difference of convex (DC) programming
Feedforward neural networks
feedforward neural networks (FFNs)
log-sum-exp (LSE) networks
Machine Learning
Mathematical analysis
Mathematical models
Mathematics
Neural networks
Numerical methods
Optimization
Optimization and Control
Subtraction
subtraction-free expressions
surrogate models
Transforms
universal approximation
LSE networks
DC programming
Subtraction-freeexpressions
Surrogate models
Feedforward neural networks
Data-driven optimization
Universal ap-proximation
ISSN:2162-237X, 2162-2388, 2162-2388
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:We show that a neural network whose output is obtained as the difference of the outputs of two feedforward networks with exponential activation function in the hidden layer and logarithmic activation function in the output node, referred to as log-sum-exp (LSE) network, is a smooth universal approximator of continuous functions over convex, compact sets. By using a logarithmic transform, this class of network maps to a family of subtraction-free ratios of generalized posynomials (GPOS), which we also show to be universal approximators of positive functions over log-convex, compact subsets of the positive orthant. The main advantage of difference-LSE networks with respect to classical feedforward neural networks is that, after a standard training phase, they provide surrogate models for a design that possesses a specific difference-of-convex-functions form, which makes them optimizable via relatively efficient numerical methods. In particular, by adapting an existing difference-of-convex algorithm to these models, we obtain an algorithm for performing an effective optimization-based design. We illustrate the proposed approach by applying it to the data-driven design of a diet for a patient with type-2 diabetes and to a nonconvex optimization problem.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2162-237X
2162-2388
2162-2388
DOI:10.1109/TNNLS.2020.2975051