Fed-DeepONet: Stochastic Gradient-Based Federated Training of Deep Operator Networks

The Deep Operator Network (DeepONet) framework is a different class of neural network architecture that one trains to learn nonlinear operators, i.e., mappings between infinite-dimensional spaces. Traditionally, DeepONets are trained using a centralized strategy that requires transferring the traini...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Algorithms Ročník 15; číslo 9; s. 325
Hlavní autoři: Moya, Christian, Lin, Guang
Médium: Journal Article
Jazyk:angličtina
Vydáno: Basel MDPI AG 01.09.2022
Témata:
Algorithms
Analysis
Clients
Communication
Computational linguistics
Computer architecture
Data security
deep learning
deep operator networks
Digital twins
Dynamic response
Dynamical systems
Engineering
Experiments
federated learning
Language processing
Natural language interfaces
Neural networks
Optimization
Pendulums
Privacy
stochastic-gradient descent
Training
ISSN:1999-4893, 1999-4893
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:The Deep Operator Network (DeepONet) framework is a different class of neural network architecture that one trains to learn nonlinear operators, i.e., mappings between infinite-dimensional spaces. Traditionally, DeepONets are trained using a centralized strategy that requires transferring the training data to a centralized location. Such a strategy, however, limits our ability to secure data privacy or use high-performance distributed/parallel computing platforms. To alleviate such limitations, in this paper, we study the federated training of DeepONets for the first time. That is, we develop a framework, which we refer to as Fed-DeepONet, that allows multiple clients to train DeepONets collaboratively under the coordination of a centralized server. To achieve Fed-DeepONets, we propose an efficient stochastic gradient-based algorithm that enables the distributed optimization of the DeepONet parameters by averaging first-order estimates of the DeepONet loss gradient. Then, to accelerate the training convergence of Fed-DeepONets, we propose a moment-enhanced (i.e., adaptive) stochastic gradient-based strategy. Finally, we verify the performance of Fed-DeepONet by learning, for different configurations of the number of clients and fractions of available clients, (i) the solution operator of a gravity pendulum and (ii) the dynamic response of a parametric library of pendulums.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
USDOE
SC0021142
ISSN:1999-4893
1999-4893
DOI:10.3390/a15090325