Deep ReLU neural networks in high-dimensional approximation

We study the computation complexity of deep ReLU (Rectified Linear Unit) neural networks for the approximation of functions from the Hölder–Zygmund space of mixed smoothness defined on the d-dimensional unit cube when the dimension d may be very large. The approximation error is measured in the norm...

Full description

Saved in:
Bibliographic Details
Published in:Neural networks Vol. 142; pp. 619 - 635
Main Authors: Dũng, Dinh, Nguyen, Van Kien
Format: Journal Article
Language:English
Published: Elsevier Ltd 01.10.2021
Subjects:
Computation complexity
Continuous piece-wise linear functions
Deep ReLU neural network
High-dimensional approximation
Hölder–Zygmund space of mixed smoothness
Sparse-grid sampling
High-dimensional approximation
Computation complexity
Deep ReLU neural network
Hölder–Zygmund space of mixed smoothness
Sparse-grid sampling
Continuous piece-wise linear functions
ISSN:0893-6080, 1879-2782, 1879-2782
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We study the computation complexity of deep ReLU (Rectified Linear Unit) neural networks for the approximation of functions from the Hölder–Zygmund space of mixed smoothness defined on the d-dimensional unit cube when the dimension d may be very large. The approximation error is measured in the norm of isotropic Sobolev space. For every function f from the Hölder–Zygmund space of mixed smoothness, we explicitly construct a deep ReLU neural network having an output that approximates f with a prescribed accuracy ɛ, and prove tight dimension-dependent upper and lower bounds of the computation complexity of the approximation, characterized as the size and depth of this deep ReLU neural network, explicitly in d and ɛ. The proof of these results in particular, relies on the approximation by sparse-grid sampling recovery based on the Faber series.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0893-6080
1879-2782
1879-2782
DOI:10.1016/j.neunet.2021.07.027