Self-adaptive physics-informed neural networks

Physics-Informed Neural Networks (PINNs) have emerged recently as a promising application of deep neural networks to the numerical solution of nonlinear partial differential equations (PDEs). However, it has been recognized that adaptive procedures are needed to force the neural network to fit accur...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	Journal of computational physics Ročník 474; s. 111722
Hlavní autori:	McClenny, Levi D., Braga-Neto, Ulisses M.
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	Elsevier Inc 01.02.2023
Predmet:	Numerical methods for PDE Physics-informed neural networks Scientific machine learning Numerical methods for PDE Physics-informed neural networks Scientific machine learning
ISSN:	0021-9991, 1090-2716
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Popis
Shrnutí:	Physics-Informed Neural Networks (PINNs) have emerged recently as a promising application of deep neural networks to the numerical solution of nonlinear partial differential equations (PDEs). However, it has been recognized that adaptive procedures are needed to force the neural network to fit accurately the stubborn spots in the solution of “stiff” PDEs. In this paper, we propose a fundamentally new way to train PINNs adaptively, where the adaptation weights are fully trainable and applied to each training point individually, so the neural network learns autonomously which regions of the solution are difficult and is forced to focus on them. The self-adaptation weights specify a soft multiplicative soft attention mask, which is reminiscent of similar mechanisms used in computer vision. The basic idea behind these SA-PINNs is to make the weights increase as the corresponding losses increase, which is accomplished by training the network to simultaneously minimize the losses and maximize the weights. In addition, we show how to build a continuous map of self-adaptive weights using Gaussian Process regression, which allows the use of stochastic gradient descent in problems where conventional gradient descent is not enough to produce accurate solutions. Finally, we derive the Neural Tangent Kernel matrix for SA-PINNs and use it to obtain a heuristic understanding of the effect of the self-adaptive weights on the dynamics of training in the limiting case of infinitely-wide PINNs, which suggests that SA-PINNs work by producing a smooth equalization of the eigenvalues of the NTK matrix corresponding to the different loss terms. In numerical experiments with several linear and nonlinear benchmark problems, the SA-PINN outperformed other state-of-the-art PINN algorithm in L2 error, while using a smaller number of training epochs. •Self-Adaptive PINNs is a new paradigm for training PINNs using trainable weights.•The SA training regime focuses on where error is the highest in the domain.•We demonstrate the use of SGD to mitigate full-batch computational limitations.•We show the SA-PINN improves gradient dynamics during training via NTK analysis.
ISSN:	0021-9991 1090-2716
DOI:	10.1016/j.jcp.2022.111722