On the capacity of deep generative networks for approximating distributions

We study the efficacy and efficiency of deep generative networks for approximating probability distributions. We prove that neural networks can transform a low-dimensional source distribution to a distribution that is arbitrarily close to a high-dimensional target distribution, when the closeness is...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Neural networks Ročník 145; s. 144 - 154
Hlavní autoři: Yang, Yunfei, Li, Zhen, Wang, Yang
Médium: Journal Article
Jazyk:angličtina
Vydáno: United States Elsevier Ltd 01.01.2022
Témata:
Approximation complexity
Benchmarking
Deep ReLU networks
Generative adversarial networks
Maximum mean discrepancy
Neural Networks, Computer
Probability
Wasserstein distance
Generative adversarial networks
Approximation complexity
Wasserstein distance
Deep ReLU networks
Maximum mean discrepancy
ISSN:0893-6080, 1879-2782, 1879-2782
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We study the efficacy and efficiency of deep generative networks for approximating probability distributions. We prove that neural networks can transform a low-dimensional source distribution to a distribution that is arbitrarily close to a high-dimensional target distribution, when the closeness is measured by Wasserstein distances and maximum mean discrepancy. Upper bounds of the approximation error are obtained in terms of the width and depth of neural network. Furthermore, it is shown that the approximation error in Wasserstein distance grows at most linearly on the ambient dimension and that the approximation order only depends on the intrinsic dimension of the target distribution. On the contrary, when f-divergences are used as metrics of distributions, the approximation property is different. We show that in order to approximate the target distribution in f-divergences, the dimension of the source distribution cannot be smaller than the intrinsic dimension of the target distribution.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0893-6080
1879-2782
1879-2782
DOI:10.1016/j.neunet.2021.10.012