Parseval Proximal Neural Networks

The aim of this paper is twofold. First, we show that a certain concatenation of a proximity operator with an affine operator is again a proximity operator on a suitable Hilbert space. Second, we use our findings to establish so-called proximal neural networks (PNNs) and stable tight frame proximal...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:The Journal of fourier analysis and applications Ročník 26; číslo 4
Hlavní autori: Hasannasab, Marzieh, Hertrich, Johannes, Neumayer, Sebastian, Plonka, Gerlind, Setzer, Simon, Steidl, Gabriele
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York Springer US 01.08.2020
Springer
Springer Nature B.V
Predmet:
Abstract Harmonic Analysis
Algorithms
Approximations and Expansions
Fourier Analysis
Hilbert space
Linear operators
Manifolds (mathematics)
Mathematical Methods in Physics
Mathematics
Mathematics and Statistics
Neural networks
Norms
Partial Differential Equations
Proximity
Shrinkage
Signal,Image and Speech Processing
Lipschitz neural networks
Proximal operators
Optimization on Stiefel manifolds
Adverserial robustness
Averaged operators
68T07
90C26
Frame shrinkage
ISSN:1069-5869, 1531-5851
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:The aim of this paper is twofold. First, we show that a certain concatenation of a proximity operator with an affine operator is again a proximity operator on a suitable Hilbert space. Second, we use our findings to establish so-called proximal neural networks (PNNs) and stable tight frame proximal neural networks. Let H and K be real Hilbert spaces, b ∈ K and T ∈ B ( H , K ) a linear operator with closed range and Moore–Penrose inverse T † . Based on the well-known characterization of proximity operators by Moreau, we prove that for any proximity operator Prox : K → K the operator T † Prox ( T · + b ) is a proximity operator on H equipped with a suitable norm. In particular, it follows for the frequently applied soft shrinkage operator Prox = S λ : ℓ 2 → ℓ 2 and any frame analysis operator T : H → ℓ 2 that the frame shrinkage operator T † S λ T is a proximity operator on a suitable Hilbert space. The concatenation of proximity operators on R d equipped with different norms establishes a PNN. If the network arises from tight frame analysis or synthesis operators, then it forms an averaged operator. In particular, it has Lipschitz constant 1 and belongs to the class of so-called Lipschitz networks, which were recently applied to defend against adversarial attacks. Moreover, due to its averaging property, PNNs can be used within so-called Plug-and-Play algorithms with convergence guarantee. In case of Parseval frames, we call the networks Parseval proximal neural networks (PPNNs). Then, the involved linear operators are in a Stiefel manifold and corresponding minimization methods can be applied for training of such networks. Finally, some proof-of-the concept examples demonstrate the performance of PPNNs.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1069-5869
1531-5851
DOI:10.1007/s00041-020-09761-7