Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations

This paper investigates the approximation properties of deep neural networks with piecewise-polynomial activation functions. We derive the required depth, width, and sparsity of a deep neural network to approximate any Hölder smooth function up to a given approximation error in Hölder norms in such...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Neural networks Ročník 161; s. 242 - 253
Hlavní autoři: Belomestny, Denis, Naumov, Alexey, Puchkin, Nikita, Samsonov, Sergey
Médium: Journal Article
Jazyk:angličtina
Vydáno: United States Elsevier Ltd 01.04.2023
Témata:
Algorithms
Approximation complexity
Deep neural networks
Hölder class
Machine Learning
Neural Networks, Computer
ReLUk activations
ReQU activations
Approximation complexity
Deep neural networks
Hölder class
ReQU activations
ReLUk activations
ReLU(k) activations
ISSN:0893-6080, 1879-2782, 1879-2782
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This paper investigates the approximation properties of deep neural networks with piecewise-polynomial activation functions. We derive the required depth, width, and sparsity of a deep neural network to approximate any Hölder smooth function up to a given approximation error in Hölder norms in such a way that all weights of this neural network are bounded by 1. The latter feature is essential to control generalization errors in many statistical and machine learning applications. •Rates and complexity for smooth function approximation in Hölder norms by ReQU neural networks.•Explicit and uniform bounds for weights of the approximating neural network.•Exponential convergence rates for analytic functions.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0893-6080
1879-2782
1879-2782
DOI:10.1016/j.neunet.2023.01.035