Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
•We employed adaptive activation functions in deep and physics-informed neural networks.•The proposed method is very simple and it is shown to accelerate convergence in neural networks.•In particular, we approximate various nonlinear functions using deep neural networks.•Physics-informed neural netw...
Gespeichert in:
| Veröffentlicht in: | Journal of computational physics Jg. 404; H. C; S. 109136 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Cambridge
Elsevier Inc
01.03.2020
Elsevier Science Ltd Elsevier |
| Schlagworte: | |
| ISSN: | 0021-9991, 1090-2716 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | •We employed adaptive activation functions in deep and physics-informed neural networks.•The proposed method is very simple and it is shown to accelerate convergence in neural networks.•In particular, we approximate various nonlinear functions using deep neural networks.•Physics-informed neural networks are employed to solve both forward and inverse problems of PDEs.•We also solved standard deep learning benchmarks problems and theoretically proved convergence results.
We employ adaptive activation functions for regression in deep and physics-informed neural networks (PINNs) to approximate smooth and discontinuous functions as well as solutions of linear and nonlinear partial differential equations. In particular, we solve the nonlinear Klein-Gordon equation, which has smooth solutions, the nonlinear Burgers equation, which can admit high gradient solutions, and the Helmholtz equation. We introduce a scalable hyper-parameter in the activation function, which can be optimized to achieve best performance of the network as it changes dynamically the topology of the loss function involved in the optimization process. The adaptive activation function has better learning capabilities than the traditional one (fixed activation) as it improves greatly the convergence rate, especially at early training, as well as the solution accuracy. To better understand the learning process, we plot the neural network solution in the frequency domain to examine how the network captures successively different frequency bands present in the solution. We consider both forward problems, where the approximate solutions are obtained, as well as inverse problems, where parameters involved in the governing equation are identified. Our simulation results show that the proposed method is a very simple and effective approach to increase the efficiency, robustness and accuracy of the neural network approximation of nonlinear functions as well as solutions of partial differential equations, especially for forward problems. We theoretically prove that in the proposed method, gradient descent algorithms are not attracted to suboptimal critical points or local minima. Furthermore, the proposed adaptive activation functions are shown to accelerate the minimization process of the loss values in standard deep learning benchmarks using CIFAR-10, CIFAR-100, SVHN, MNIST, KMNIST, Fashion-MNIST, and Semeion datasets with and without data augmentation. |
|---|---|
| AbstractList | We employ adaptive activation functions for regression in deep and physics-informed neural networks (PINNs) to approximate smooth and discontinuous functions as well as solutions of linear and nonlinear partial differential equations. In particular, we solve the nonlinear Klein-Gordon equation, which has smooth solutions, the nonlinear Burgers equation, which can admit high gradient solutions, and the Helmholtz equation. We introduce a scalable hyper-parameter in the activation function, which can be optimized to achieve best performance of the network as it changes dynamically the topology of the loss function involved in the optimization process. The adaptive activation function has better learning capabilities than the traditional one (fixed activation) as it improves greatly the convergence rate, especially at early training, as well as the solution accuracy. To better understand the learning process, we plot the neural network solution in the frequency domain to examine how the network captures successively different frequency bands present in the solution. We consider both forward problems, where the approximate solutions are obtained, as well as inverse problems, where parameters involved in the governing equation are identified. Our simulation results show that the proposed method is a very simple and effective approach to increase the efficiency, robustness and accuracy of the neural network approximation of nonlinear functions as well as solutions of partial differential equations, especially for forward problems. We theoretically prove that in the proposed method, gradient descent algorithms are not attracted to suboptimal critical points or local minima. Furthermore, the proposed adaptive activation functions are shown to accelerate the minimization process of the loss values in standard deep learning benchmarks using CIFAR-10, CIFAR-100, SVHN, MNIST, KMNIST, Fashion-MNIST, and Semeion datasets with and without data augmentation. •We employed adaptive activation functions in deep and physics-informed neural networks.•The proposed method is very simple and it is shown to accelerate convergence in neural networks.•In particular, we approximate various nonlinear functions using deep neural networks.•Physics-informed neural networks are employed to solve both forward and inverse problems of PDEs.•We also solved standard deep learning benchmarks problems and theoretically proved convergence results. We employ adaptive activation functions for regression in deep and physics-informed neural networks (PINNs) to approximate smooth and discontinuous functions as well as solutions of linear and nonlinear partial differential equations. In particular, we solve the nonlinear Klein-Gordon equation, which has smooth solutions, the nonlinear Burgers equation, which can admit high gradient solutions, and the Helmholtz equation. We introduce a scalable hyper-parameter in the activation function, which can be optimized to achieve best performance of the network as it changes dynamically the topology of the loss function involved in the optimization process. The adaptive activation function has better learning capabilities than the traditional one (fixed activation) as it improves greatly the convergence rate, especially at early training, as well as the solution accuracy. To better understand the learning process, we plot the neural network solution in the frequency domain to examine how the network captures successively different frequency bands present in the solution. We consider both forward problems, where the approximate solutions are obtained, as well as inverse problems, where parameters involved in the governing equation are identified. Our simulation results show that the proposed method is a very simple and effective approach to increase the efficiency, robustness and accuracy of the neural network approximation of nonlinear functions as well as solutions of partial differential equations, especially for forward problems. We theoretically prove that in the proposed method, gradient descent algorithms are not attracted to suboptimal critical points or local minima. Furthermore, the proposed adaptive activation functions are shown to accelerate the minimization process of the loss values in standard deep learning benchmarks using CIFAR-10, CIFAR-100, SVHN, MNIST, KMNIST, Fashion-MNIST, and Semeion datasets with and without data augmentation. |
| ArticleNumber | 109136 |
| Author | Kawaguchi, Kenji Jagtap, Ameya D. Karniadakis, George Em |
| Author_xml | – sequence: 1 givenname: Ameya D. orcidid: 0000-0002-8831-1000 surname: Jagtap fullname: Jagtap, Ameya D. email: ameya_jagtap@brown.edu organization: Division of Applied Mathematics, Brown University, 182 George Street, Providence, RI 02912, USA – sequence: 2 givenname: Kenji surname: Kawaguchi fullname: Kawaguchi, Kenji email: kawaguch@mit.edu organization: Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA – sequence: 3 givenname: George Em surname: Karniadakis fullname: Karniadakis, George Em email: george_karniadakis@brown.edu organization: Division of Applied Mathematics, Brown University, 182 George Street, Providence, RI 02912, USA |
| BackLink | https://www.osti.gov/biblio/1775904$$D View this record in Osti.gov |
| BookMark | eNp9kDtv3DAQhInABnJ-_AB3QlLrsqSeRCrDiBMDBtIkNcFbrmwqd6RCUhf435uyUrm4anYXM4vBd8HOnHfE2A2HLQfefhm3I05bAVzmXfKq_cA2eYBSdLw9YxsAwUspJf_ILmIcAaBv6n7D9K3RU7JHKjRm0cl6Vwyzw2WI-Yi0p6ATFejdkcITOaTCusIQTYV2ppieX6LFWFo3-HAgUziag95nSf98-BOv2Pmg95Gu_-sl-33_7dfdj_Lx5_eHu9vHEuuWp9IIAN3UJBrQpgZEQBIVaoma923XybaRZodtxetBi0FwGKDiXV_LXUPNrq8u2af1r4_Jqog2ET7n0o4wKd51jYQ6mz6vpin4vzPFpEY_B5d7KVE1FVR1--biqwuDjzHQoKZgDzq8KA5qwa1GlXGrBbdacedM9y6TG7zhTEHb_cnk1zVJmc7RUljKL5iNDUt34-2J9Csd-pxj |
| CitedBy_id | crossref_primary_10_1007_s11071_023_08641_1 crossref_primary_10_1080_00207160_2022_2128674 crossref_primary_10_1016_j_camwa_2025_06_024 crossref_primary_10_1063_5_0273148 crossref_primary_10_1016_j_cpc_2025_109782 crossref_primary_10_1038_s41598_023_40766_6 crossref_primary_10_1016_j_jcp_2022_111592 crossref_primary_10_3390_rs16010180 crossref_primary_10_1007_s00366_024_01944_w crossref_primary_10_1016_j_rinam_2025_100539 crossref_primary_10_1016_j_aei_2025_103456 crossref_primary_10_3390_app15148092 crossref_primary_10_7554_eLife_91911_3 crossref_primary_10_1007_s00466_023_02365_0 crossref_primary_10_1016_j_oceaneng_2024_118341 crossref_primary_10_3390_en15186823 crossref_primary_10_1007_s11831_023_09890_4 crossref_primary_10_1016_j_jcp_2020_109951 crossref_primary_10_1016_j_chaos_2025_116964 crossref_primary_10_1016_j_jmmm_2021_168951 crossref_primary_10_3390_math11143081 crossref_primary_10_1016_j_apenergy_2021_116641 crossref_primary_10_1016_j_rineng_2023_101023 crossref_primary_10_1016_j_engfracmech_2025_111026 crossref_primary_10_1016_j_cma_2024_117290 crossref_primary_10_1016_j_jmps_2023_105416 crossref_primary_10_1007_s10921_025_01241_6 crossref_primary_10_1109_ACCESS_2024_3504962 crossref_primary_10_1016_j_geoen_2023_211982 crossref_primary_10_1016_j_engstruct_2022_115484 crossref_primary_10_1016_j_ijheatmasstransfer_2023_124805 crossref_primary_10_1007_s11071_024_09648_y crossref_primary_10_1016_j_apenergy_2023_122439 crossref_primary_10_1016_j_ijheatmasstransfer_2020_120743 crossref_primary_10_1007_s11071_024_10309_3 crossref_primary_10_1016_j_jcp_2025_114226 crossref_primary_10_1089_3dp_2022_0363 crossref_primary_10_1016_j_geoen_2025_214010 crossref_primary_10_5194_gchron_6_227_2024 crossref_primary_10_1038_s43588_023_00412_7 crossref_primary_10_1016_j_neucom_2025_129978 crossref_primary_10_1109_TED_2023_3316635 crossref_primary_10_1016_j_aej_2025_05_072 crossref_primary_10_1016_j_ymssp_2024_112009 crossref_primary_10_1080_00295639_2022_2123211 crossref_primary_10_1016_j_neucom_2024_129134 crossref_primary_10_3390_sym17091441 crossref_primary_10_7554_eLife_91911 crossref_primary_10_1061__ASCE_EM_1943_7889_0002156 crossref_primary_10_1016_j_ijfatigue_2024_108382 crossref_primary_10_1007_s00366_023_01914_8 crossref_primary_10_1007_s12572_023_00365_0 crossref_primary_10_1016_j_jcp_2022_111121 crossref_primary_10_1016_j_cma_2024_117276 crossref_primary_10_1016_j_cma_2024_117036 crossref_primary_10_1016_j_cpc_2025_109753 crossref_primary_10_1016_j_engappai_2023_107324 crossref_primary_10_3390_en15249347 crossref_primary_10_1016_j_cpc_2025_109757 crossref_primary_10_1109_TPS_2024_3404074 crossref_primary_10_1088_1402_4896_ad7dc0 crossref_primary_10_1007_s11071_025_11457_w crossref_primary_10_1016_j_engappai_2023_107453 crossref_primary_10_1016_j_jcp_2023_112624 crossref_primary_10_1063_5_0221924 crossref_primary_10_1016_j_physleta_2025_130998 crossref_primary_10_1016_j_jcp_2022_111232 crossref_primary_10_3390_app14135490 crossref_primary_10_1016_j_jcp_2022_111270 crossref_primary_10_3390_a15120447 crossref_primary_10_1016_j_jcp_2024_113299 crossref_primary_10_1137_23M1622696 crossref_primary_10_1016_j_matcom_2024_10_039 crossref_primary_10_1007_s00521_021_06373_0 crossref_primary_10_1016_j_engappai_2023_107773 crossref_primary_10_1007_s11071_024_10497_y crossref_primary_10_1016_j_finel_2024_104305 crossref_primary_10_3390_a16060305 crossref_primary_10_1016_j_cma_2024_117381 crossref_primary_10_1016_j_enganabound_2024_105905 crossref_primary_10_1007_s11071_022_07803_x crossref_primary_10_1007_s12190_025_02588_9 crossref_primary_10_1126_sciadv_ads5236 crossref_primary_10_1007_s11071_025_10871_4 crossref_primary_10_3390_math11112529 crossref_primary_10_1016_j_jcp_2022_111024 crossref_primary_10_1016_j_camwa_2023_09_047 crossref_primary_10_1016_j_jcp_2022_111260 crossref_primary_10_1016_j_cma_2025_118403 crossref_primary_10_1016_j_cpc_2025_109572 crossref_primary_10_1016_j_jappgeo_2024_105479 crossref_primary_10_1186_s13104_025_07142_1 crossref_primary_10_1088_1402_4896_acd307 crossref_primary_10_1016_j_jcp_2024_113284 crossref_primary_10_1137_23M1621356 crossref_primary_10_1016_j_engappai_2023_107307 crossref_primary_10_1088_0256_307X_38_3_038701 crossref_primary_10_1016_j_ijfatigue_2024_108678 crossref_primary_10_1016_j_asoc_2021_108050 crossref_primary_10_1103_PhysRevApplied_22_L031002 crossref_primary_10_1016_j_camwa_2023_09_030 crossref_primary_10_1109_JLT_2024_3401419 crossref_primary_10_1016_j_neunet_2025_107375 crossref_primary_10_1088_1873_7005_adb32e crossref_primary_10_1109_ACCESS_2023_3297724 crossref_primary_10_1016_j_engappai_2023_106425 crossref_primary_10_3390_app12041830 crossref_primary_10_1016_j_engappai_2022_105176 crossref_primary_10_3390_e25040674 crossref_primary_10_1016_j_ijheatmasstransfer_2022_123420 crossref_primary_10_1108_COMPEL_12_2022_0436 crossref_primary_10_1016_j_cma_2024_117000 crossref_primary_10_1016_j_cpc_2025_109569 crossref_primary_10_1007_s00366_023_01881_0 crossref_primary_10_3390_math12233873 crossref_primary_10_3390_math11081843 crossref_primary_10_1016_j_chaos_2023_114090 crossref_primary_10_1016_j_jmps_2023_105231 crossref_primary_10_1007_s00466_024_02559_0 crossref_primary_10_1016_j_jocs_2025_102608 crossref_primary_10_1016_j_ymssp_2025_112561 crossref_primary_10_1016_j_engappai_2023_107528 crossref_primary_10_1038_s43588_021_00158_0 crossref_primary_10_1109_ACCESS_2020_2987324 crossref_primary_10_1016_j_cma_2025_117897 crossref_primary_10_1016_j_cpc_2025_109672 crossref_primary_10_1016_j_jweia_2024_105935 crossref_primary_10_1007_s00521_023_08876_4 crossref_primary_10_1016_j_eswa_2025_128977 crossref_primary_10_1140_epjp_s13360_025_06644_y crossref_primary_10_1038_s41598_025_02978_w crossref_primary_10_1016_j_trc_2024_104658 crossref_primary_10_1186_s42774_021_00094_7 crossref_primary_10_1016_j_enganabound_2022_09_024 crossref_primary_10_1016_j_cma_2022_115491 crossref_primary_10_1016_j_ijsolstr_2023_112321 crossref_primary_10_1016_j_jcp_2023_112309 crossref_primary_10_1109_TNNLS_2023_3247163 crossref_primary_10_1016_j_jocs_2024_102318 crossref_primary_10_1088_1572_9494_ac2055 crossref_primary_10_1007_s11071_023_08654_w crossref_primary_10_1007_s11424_024_3349_z crossref_primary_10_1016_j_camwa_2022_07_002 crossref_primary_10_1016_j_jcp_2022_111277 crossref_primary_10_3390_math13111882 crossref_primary_10_1007_s00366_023_01883_y crossref_primary_10_1016_j_cpc_2024_109130 crossref_primary_10_1002_advs_202300439 crossref_primary_10_3390_math12111617 crossref_primary_10_1093_imanum_drab032 crossref_primary_10_3390_electronics12122570 crossref_primary_10_3150_24_BEJ1799 crossref_primary_10_1016_j_advwatres_2024_104731 crossref_primary_10_1038_s41598_021_96723_8 crossref_primary_10_1007_s11082_023_04950_2 crossref_primary_10_2514_1_J064064 crossref_primary_10_1007_s11831_023_09954_5 crossref_primary_10_1016_j_cpc_2024_109428 crossref_primary_10_3934_math_2025972 crossref_primary_10_1016_j_engappai_2025_111554 crossref_primary_10_1016_j_cnsns_2024_108242 crossref_primary_10_1016_j_neunet_2025_107983 crossref_primary_10_1007_s12596_024_01831_z crossref_primary_10_1016_j_cma_2024_116805 crossref_primary_10_1016_j_geoen_2023_212320 crossref_primary_10_1016_j_engappai_2025_110347 crossref_primary_10_1016_j_jcp_2021_110498 crossref_primary_10_1016_j_camwa_2024_09_007 crossref_primary_10_1063_5_0264041 crossref_primary_10_1016_j_compfluid_2023_106114 crossref_primary_10_3390_app14167002 crossref_primary_10_3390_electronics12143146 crossref_primary_10_1109_TGRS_2024_3371528 crossref_primary_10_1007_s11082_023_04645_8 crossref_primary_10_1016_j_enganabound_2024_03_030 crossref_primary_10_1109_ACCESS_2020_2981044 crossref_primary_10_1016_j_jqsrt_2021_107705 crossref_primary_10_1109_TMI_2022_3161653 crossref_primary_10_1007_s13131_024_2329_4 crossref_primary_10_1016_j_swevo_2024_101589 crossref_primary_10_1017_jfm_2023_147 crossref_primary_10_1080_00295639_2023_2236840 crossref_primary_10_1016_j_engappai_2023_106867 crossref_primary_10_1016_j_engfracmech_2025_111319 crossref_primary_10_1088_1572_9494_accb8d crossref_primary_10_1007_s00158_022_03425_4 crossref_primary_10_1016_j_oceaneng_2024_120239 crossref_primary_10_1080_14685248_2023_2274100 crossref_primary_10_1016_j_cnsns_2024_108229 crossref_primary_10_1007_s11424_024_3467_7 crossref_primary_10_1016_j_neucom_2021_10_036 crossref_primary_10_1016_j_jcp_2022_111769 crossref_primary_10_1063_5_0216609 crossref_primary_10_1016_j_apenergy_2024_123179 crossref_primary_10_3390_math11092016 crossref_primary_10_1016_j_cma_2024_116907 crossref_primary_10_1038_s41598_022_08745_5 crossref_primary_10_1016_j_cma_2022_115852 crossref_primary_10_1016_j_cma_2024_116906 crossref_primary_10_1007_s00521_022_07294_2 crossref_primary_10_3390_fluids9100231 crossref_primary_10_1007_s10489_024_05461_7 crossref_primary_10_1016_j_jcp_2022_111402 crossref_primary_10_3390_math13152344 crossref_primary_10_1016_j_enganabound_2025_106200 crossref_primary_10_1016_j_probengmech_2022_103240 crossref_primary_10_3934_acse_2025019 crossref_primary_10_1137_21M1460764 crossref_primary_10_3390_app15158740 crossref_primary_10_1088_2632_2153_acb416 crossref_primary_10_1109_MSP_2021_3118904 crossref_primary_10_1088_2632_2153_ad2973 crossref_primary_10_1007_s10967_022_08735_x crossref_primary_10_1007_s10409_021_01144_5 crossref_primary_10_1016_j_euromechsol_2021_104225 crossref_primary_10_1109_ACCESS_2024_3495732 crossref_primary_10_1016_j_biosystemseng_2023_04_012 crossref_primary_10_1016_j_enganabound_2025_106207 crossref_primary_10_1007_s10462_022_10329_8 crossref_primary_10_1371_journal_pone_0315762 crossref_primary_10_3390_fluids7050154 crossref_primary_10_1109_ACCESS_2024_3481671 crossref_primary_10_1016_j_cma_2025_118356 crossref_primary_10_1016_j_neucom_2020_09_006 crossref_primary_10_1088_1572_9494_ad1a0e crossref_primary_10_3390_app14020859 crossref_primary_10_1109_ACCESS_2024_3422224 crossref_primary_10_1016_j_knosys_2024_111831 crossref_primary_10_1080_10589759_2024_2350575 crossref_primary_10_1016_j_jcp_2023_111944 crossref_primary_10_1016_j_jcp_2022_111789 crossref_primary_10_1002_tal_70002 crossref_primary_10_3390_diagnostics12020557 crossref_primary_10_1017_jfm_2025_91 crossref_primary_10_1016_j_cma_2025_118125 crossref_primary_10_1142_S0219455425501986 crossref_primary_10_1007_s44379_025_00015_1 crossref_primary_10_1007_s12206_024_0624_9 crossref_primary_10_1016_j_compstruc_2022_106761 crossref_primary_10_1016_j_geoen_2024_212938 crossref_primary_10_1016_j_ijheatmasstransfer_2025_126980 crossref_primary_10_1016_j_est_2023_107037 crossref_primary_10_1016_j_compfluid_2023_106164 crossref_primary_10_1063_5_0279565 crossref_primary_10_1007_s11424_024_3321_y crossref_primary_10_1088_2634_4386_ac999b crossref_primary_10_1016_j_compgeo_2025_107348 crossref_primary_10_1016_j_camwa_2025_01_025 crossref_primary_10_3390_math10121976 crossref_primary_10_1016_j_engappai_2023_105852 crossref_primary_10_3390_math13111801 crossref_primary_10_1016_j_jcp_2022_111690 crossref_primary_10_1029_2024WR037786 crossref_primary_10_3389_arc_2025_14842 crossref_primary_10_1016_j_neucom_2022_03_008 crossref_primary_10_1007_s10773_025_06034_1 crossref_primary_10_1016_j_jcp_2025_114370 crossref_primary_10_1016_j_cma_2025_118229 crossref_primary_10_1109_MCI_2021_3061854 crossref_primary_10_1162_neco_a_01462 crossref_primary_10_1016_j_cam_2023_115369 crossref_primary_10_3390_math10224203 crossref_primary_10_1016_j_inffus_2025_103255 crossref_primary_10_3390_s23198205 crossref_primary_10_1080_17486025_2025_2502029 crossref_primary_10_1109_TIM_2023_3334377 crossref_primary_10_1016_j_cma_2024_117075 crossref_primary_10_1007_s44379_025_00016_0 crossref_primary_10_1016_j_jocs_2025_102577 crossref_primary_10_1109_TRO_2024_3411850 crossref_primary_10_1016_j_jocs_2025_102575 crossref_primary_10_1007_s00366_025_02174_4 crossref_primary_10_1016_j_autcon_2025_106246 crossref_primary_10_1016_j_enganabound_2025_106363 crossref_primary_10_1016_j_engappai_2024_108085 crossref_primary_10_1016_j_euromechsol_2022_104849 crossref_primary_10_1016_j_cma_2023_115944 crossref_primary_10_1016_j_enbuild_2024_114071 crossref_primary_10_1016_j_jcp_2025_114122 crossref_primary_10_1111_jfpp_16949 crossref_primary_10_1002_nme_7296 crossref_primary_10_1109_TPAMI_2023_3307688 crossref_primary_10_3390_mca28020052 crossref_primary_10_1016_j_cma_2023_116012 crossref_primary_10_1063_5_0227921 crossref_primary_10_1007_s10409_024_24140_x crossref_primary_10_1109_ACCESS_2024_3359417 crossref_primary_10_1016_j_engappai_2023_105837 crossref_primary_10_1063_5_0220173 crossref_primary_10_1016_j_cageo_2024_105611 crossref_primary_10_1016_j_camwa_2024_08_035 crossref_primary_10_1016_j_istruc_2023_01_115 crossref_primary_10_1088_1361_6501_acc049 crossref_primary_10_1038_s41598_022_16463_1 crossref_primary_10_1007_s10915_022_01980_y crossref_primary_10_1137_20M1366587 crossref_primary_10_1007_s10921_020_00705_1 crossref_primary_10_1016_j_ijfatigue_2022_107270 crossref_primary_10_1016_j_jcp_2021_110651 crossref_primary_10_1093_jge_gxac048 crossref_primary_10_1007_s10915_022_01939_z crossref_primary_10_1016_j_neunet_2023_08_014 crossref_primary_10_3390_app11209411 crossref_primary_10_1016_j_cma_2024_116883 crossref_primary_10_1137_23M1583375 crossref_primary_10_1016_j_jcp_2023_112291 crossref_primary_10_1109_ACCESS_2024_3402240 crossref_primary_10_1016_j_autcon_2022_104219 crossref_primary_10_1016_j_ymssp_2024_111920 crossref_primary_10_1007_s11071_023_08614_4 crossref_primary_10_1016_j_engappai_2022_105790 crossref_primary_10_1007_s00366_024_02038_3 crossref_primary_10_1007_s11600_022_00740_8 crossref_primary_10_1016_j_cma_2024_116996 crossref_primary_10_1016_j_camwa_2024_01_021 crossref_primary_10_1016_j_neucom_2025_130167 crossref_primary_10_1016_j_cma_2023_116278 crossref_primary_10_1137_21M1447039 crossref_primary_10_3390_mca30050094 crossref_primary_10_1016_j_jfoodeng_2023_111774 crossref_primary_10_1109_TGRS_2023_3295414 crossref_primary_10_1016_j_camwa_2023_05_004 crossref_primary_10_1029_2021JB023120 crossref_primary_10_1088_2632_2153_ad450f crossref_primary_10_3390_bdcc9020025 crossref_primary_10_1016_j_jcp_2025_113905 crossref_primary_10_1016_j_isatra_2024_11_049 crossref_primary_10_23939_mmc2025_02_549 crossref_primary_10_1007_s10915_024_02761_5 crossref_primary_10_1007_s10462_025_11322_7 crossref_primary_10_1109_ACCESS_2024_3354058 crossref_primary_10_3390_biomedicines12020395 crossref_primary_10_1016_j_heliyon_2024_e40799 crossref_primary_10_1134_S1995080223010213 crossref_primary_10_3389_fspas_2021_732275 crossref_primary_10_1016_j_anucene_2022_109234 crossref_primary_10_1038_s42005_024_01521_z crossref_primary_10_1007_s10409_021_01148_1 crossref_primary_10_1016_j_inffus_2023_102041 crossref_primary_10_1016_j_neunet_2024_106408 crossref_primary_10_1016_j_tws_2024_112495 crossref_primary_10_1002_nme_7377 crossref_primary_10_1016_j_petsci_2025_07_008 crossref_primary_10_1016_j_engappai_2024_108280 crossref_primary_10_1016_j_jcp_2021_110884 crossref_primary_10_1016_j_cma_2023_116299 crossref_primary_10_1016_j_jcp_2024_112761 crossref_primary_10_1016_j_jmps_2024_105570 crossref_primary_10_1088_1572_9494_adcc8e crossref_primary_10_1007_s10409_023_22438_x crossref_primary_10_2514_1_J063570 crossref_primary_10_1016_j_neucom_2024_127529 crossref_primary_10_1007_s13226_024_00541_3 crossref_primary_10_1007_s44379_025_00035_x crossref_primary_10_1137_19M1274067 crossref_primary_10_1016_j_cma_2021_114424 crossref_primary_10_1016_j_neucom_2025_131589 crossref_primary_10_1109_TAP_2025_3571529 crossref_primary_10_1007_s00202_024_02432_6 crossref_primary_10_1016_j_commatsci_2023_112583 crossref_primary_10_1016_j_wavemoti_2021_102823 crossref_primary_10_1016_j_jhydrol_2025_133624 crossref_primary_10_1137_22M1517081 crossref_primary_10_3390_computation9090097 crossref_primary_10_1007_s00466_023_02434_4 crossref_primary_10_3390_cancers14143442 crossref_primary_10_1007_s11431_022_2118_9 crossref_primary_10_3390_s24061829 crossref_primary_10_1016_j_camwa_2024_07_024 crossref_primary_10_1016_j_jcp_2024_112865 crossref_primary_10_1016_j_physd_2022_133430 crossref_primary_10_1016_j_jtice_2023_105318 crossref_primary_10_1016_j_camwa_2024_06_008 crossref_primary_10_1016_j_firesaf_2025_104379 crossref_primary_10_1016_j_compfluid_2025_106700 crossref_primary_10_1190_geo2022_0479_1 crossref_primary_10_3390_rs15122986 crossref_primary_10_1016_j_apm_2023_07_011 crossref_primary_10_1016_j_ijheatmasstransfer_2023_124392 crossref_primary_10_1016_j_chaos_2022_112182 crossref_primary_10_1038_s41598_023_29806_3 crossref_primary_10_3390_app10175917 crossref_primary_10_1016_j_jcp_2021_110318 crossref_primary_10_3390_app13126892 crossref_primary_10_1016_j_advwatres_2025_105001 crossref_primary_10_1016_j_jcp_2024_113709 crossref_primary_10_5194_hess_26_4469_2022 crossref_primary_10_1016_j_fuel_2025_136475 crossref_primary_10_1016_j_physd_2023_133945 crossref_primary_10_1016_j_neucom_2025_131440 crossref_primary_10_1080_27684520_2025_2542577 crossref_primary_10_3390_s23146649 crossref_primary_10_1016_j_engappai_2022_105724 crossref_primary_10_1007_s13160_023_00577_8 crossref_primary_10_1007_s00419_024_02622_5 crossref_primary_10_1080_10407790_2023_2264489 crossref_primary_10_1093_imanum_drae081 crossref_primary_10_1029_2022WR034122 crossref_primary_10_1016_j_cma_2020_113028 crossref_primary_10_1016_j_cma_2024_116819 crossref_primary_10_1016_j_jcp_2021_110683 crossref_primary_10_1016_j_cma_2024_116813 crossref_primary_10_1016_j_neunet_2025_107831 crossref_primary_10_1016_j_neucom_2025_131318 crossref_primary_10_1016_j_camwa_2022_02_004 crossref_primary_10_1016_j_camwa_2024_06_029 crossref_primary_10_1007_s11340_024_01139_w crossref_primary_10_1016_j_ijheatfluidflow_2025_110011 crossref_primary_10_23939_cds2024_03_139 crossref_primary_10_3390_pr9050737 crossref_primary_10_1186_s40323_024_00273_3 crossref_primary_10_1016_j_physd_2024_134399 crossref_primary_10_3390_info14020065 crossref_primary_10_1093_mnras_stad2840 crossref_primary_10_3390_agriculture13010124 crossref_primary_10_1108_HFF_09_2023_0568 crossref_primary_10_3390_s23167081 crossref_primary_10_1016_j_enconman_2023_117033 crossref_primary_10_1016_j_ijmecsci_2024_109525 crossref_primary_10_1016_j_cma_2024_117691 crossref_primary_10_1038_s41598_019_54707_9 crossref_primary_10_1063_5_0284425 crossref_primary_10_1016_j_neunet_2024_106703 crossref_primary_10_1016_j_ijfatigue_2025_108933 crossref_primary_10_1016_j_patcog_2022_108988 crossref_primary_10_1073_pnas_2102721118 crossref_primary_10_1080_24725579_2024_2398592 crossref_primary_10_1016_j_neucom_2022_06_111 crossref_primary_10_1063_5_0232675 crossref_primary_10_1016_j_neunet_2025_108027 crossref_primary_10_1007_s11071_024_10655_2 crossref_primary_10_1016_j_compchemeng_2022_107898 crossref_primary_10_1016_j_jcp_2023_112464 crossref_primary_10_1016_j_cma_2023_116674 crossref_primary_10_1007_s00466_023_02291_1 crossref_primary_10_1016_j_medengphy_2024_104277 crossref_primary_10_1088_1402_4896_ad55be crossref_primary_10_1016_j_cnsns_2025_109049 crossref_primary_10_3390_sym16111490 crossref_primary_10_1088_2399_6528_ace416 crossref_primary_10_1109_TMI_2021_3123300 crossref_primary_10_1016_j_cnsns_2022_107051 crossref_primary_10_1007_s10773_023_05365_1 crossref_primary_10_1016_j_ijmecsci_2024_109783 crossref_primary_10_1016_j_cma_2023_116561 crossref_primary_10_12677_aam_2025_143106 crossref_primary_10_1016_j_jcp_2024_113341 crossref_primary_10_1002_nme_7406 crossref_primary_10_1016_j_jcp_2021_110928 crossref_primary_10_1093_pnasnexus_pgae005 crossref_primary_10_1016_j_engappai_2022_104953 crossref_primary_10_1137_23M1549870 crossref_primary_10_1016_j_tws_2023_111423 crossref_primary_10_1016_j_euromechflu_2025_204307 crossref_primary_10_1016_j_euromechflu_2025_204306 crossref_primary_10_1016_j_neucom_2024_127240 crossref_primary_10_1016_j_oceaneng_2022_110775 crossref_primary_10_1016_j_chaos_2024_115727 crossref_primary_10_1016_j_jcp_2024_113698 crossref_primary_10_1063_5_0218482 crossref_primary_10_1109_TII_2024_3452203 crossref_primary_10_1016_j_physd_2024_134066 crossref_primary_10_1016_j_jcp_2023_112242 crossref_primary_10_1016_j_mtcomm_2021_102719 crossref_primary_10_1016_j_dsp_2024_104766 crossref_primary_10_1038_s41598_024_62117_9 crossref_primary_10_1016_j_engappai_2023_106049 crossref_primary_10_1007_s11071_024_10786_6 crossref_primary_10_1016_j_jcp_2023_112360 crossref_primary_10_1111_mice_12677 crossref_primary_10_32890_jict_20_1_2021_9267 crossref_primary_10_1016_j_autcon_2025_105983 crossref_primary_10_1016_j_camwa_2025_07_037 crossref_primary_10_1177_20552076241232882 crossref_primary_10_3390_automation3030021 crossref_primary_10_1016_j_engappai_2025_112044 crossref_primary_10_1007_s00466_024_02554_5 crossref_primary_10_1016_j_neunet_2024_106732 crossref_primary_10_1016_j_cep_2023_109540 crossref_primary_10_3390_fluids8020046 crossref_primary_10_1007_s13160_023_00617_3 crossref_primary_10_1016_j_jcp_2024_113561 crossref_primary_10_1093_mnras_stac3512 crossref_primary_10_3390_s23146346 crossref_primary_10_1007_s40314_025_03295_8 crossref_primary_10_1016_j_ijheatmasstransfer_2025_127040 crossref_primary_10_1111_mice_12685 crossref_primary_10_1007_s42967_024_00398_7 crossref_primary_10_1016_j_jcp_2021_110624 crossref_primary_10_1016_j_compstruc_2025_107839 crossref_primary_10_1080_01495739_2024_2321205 crossref_primary_10_1038_s41598_021_99037_x crossref_primary_10_1007_s11071_023_09173_4 crossref_primary_10_1007_s40314_023_02323_9 crossref_primary_10_1016_j_cma_2024_117405 crossref_primary_10_1088_1674_1056_ad21f3 crossref_primary_10_1063_5_0276518 crossref_primary_10_1109_ACCESS_2021_3140189 crossref_primary_10_1063_5_0286561 crossref_primary_10_1016_j_ocemod_2025_102601 crossref_primary_10_1007_s11071_021_06819_z crossref_primary_10_1016_j_rineng_2025_105440 crossref_primary_10_1007_s11082_023_04934_2 crossref_primary_10_1080_17499518_2024_2315301 crossref_primary_10_1021_acs_jctc_4c01747 crossref_primary_10_3390_batteries8100145 crossref_primary_10_1007_s00170_023_11543_6 crossref_primary_10_1016_j_cma_2025_117914 crossref_primary_10_1103_PhysRevE_111_025301 crossref_primary_10_1109_TNNLS_2025_3545967 crossref_primary_10_1080_17455030_2023_2192818 crossref_primary_10_3390_rs17010124 crossref_primary_10_1371_journal_pone_0261571 crossref_primary_10_1016_j_ijepes_2021_107399 crossref_primary_10_1007_s11071_021_06554_5 crossref_primary_10_1038_s41598_022_11058_2 crossref_primary_10_1109_TGRS_2023_3307750 crossref_primary_10_1016_j_engappai_2023_107489 crossref_primary_10_1016_j_ijmultiphaseflow_2024_104861 crossref_primary_10_1155_2023_2973249 crossref_primary_10_1007_s11063_024_11644_7 crossref_primary_10_1063_5_0232852 crossref_primary_10_1093_imatrm_tnac001 crossref_primary_10_1016_j_jcp_2025_113837 crossref_primary_10_1016_j_cma_2025_117806 crossref_primary_10_1088_1742_6596_2866_1_012053 crossref_primary_10_1016_j_jcp_2024_113656 crossref_primary_10_1038_s41467_022_34922_1 crossref_primary_10_3390_s23146371 |
| Cites_doi | 10.1016/S0065-2156(08)70100-5 10.1137/17M1120762 10.1109/72.712178 10.1016/j.jcp.2019.01.036 10.1016/j.jcp.2017.11.039 10.1175/1520-0493(1915)43<163:SRROTM>2.0.CO;2 10.1137/140974596 10.1126/sciadv.1602614 10.1016/j.jcp.2017.07.050 10.1016/j.jcp.2015.11.012 10.1007/BF02724733 10.1016/0045-7930(86)90036-8 10.1017/jfm.2018.872 10.1016/j.jcp.2018.10.045 10.1016/j.jcp.2017.01.060 10.1007/s11063-004-0637-4 10.1109/5.726791 10.1016/j.neucom.2017.06.070 10.1109/72.279181 10.1016/j.cnsns.2006.08.005 10.1007/s00521-012-0858-9 |
| ContentType | Journal Article |
| Copyright | 2019 Elsevier Inc. Copyright Elsevier Science Ltd. Mar 1, 2020 |
| Copyright_xml | – notice: 2019 Elsevier Inc. – notice: Copyright Elsevier Science Ltd. Mar 1, 2020 |
| DBID | AAYXX CITATION 7SC 7SP 7U5 8FD JQ2 L7M L~C L~D OTOTI |
| DOI | 10.1016/j.jcp.2019.109136 |
| DatabaseName | CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Solid State and Superconductivity Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional OSTI.GOV |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Solid State and Superconductivity Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences |
| EISSN | 1090-2716 |
| ExternalDocumentID | 1775904 10_1016_j_jcp_2019_109136 S0021999119308411 |
| GroupedDBID | --K --M -~X .~1 0R~ 1B1 1RT 1~. 1~5 4.4 457 4G. 5GY 5VS 6OB 7-5 71M 8P~ 9JN AABNK AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAXUO AAYFN ABBOA ABFRF ABJNI ABMAC ABNEU ABYKQ ACBEA ACDAQ ACFVG ACGFO ACGFS ACNCT ACRLP ACZNC ADBBV ADEZE AEBSH AEFWE AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHZHX AIALX AIEXJ AIKHN AITUG AIVDX AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD AXJTR BKOJK BLXMC CS3 DM4 DU5 EBS EFBJH EFLBG EO8 EO9 EP2 EP3 F5P FDB FEDTE FIRID FNPLU FYGXN G-Q GBLVA GBOLZ HLZ HVGLF IHE J1W K-O KOM LG5 LX9 LZ4 M37 M41 MO0 N9A O-L O9- OAUVE OGIMB OZT P-8 P-9 P2P PC. Q38 RNS ROL RPZ SDF SDG SDP SES SPC SPCBC SPD SSQ SSV SSZ T5K TN5 UPT YQT ZMT ZU3 ~02 ~G- 29K 6TJ 8WZ 9DU A6W AAQXK AATTM AAXKI AAYWO AAYXX ABFNM ABWVN ABXDB ACLOT ACNNM ACRPL ACVFH ADCNI ADFGL ADIYS ADJOM ADMUD ADNMO AEIPS AEUPX AFFNX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP ASPBG AVWKF AZFZN BBWZM CAG CITATION COF D-I EFKBS EJD FGOYB G-2 HME HMV HZ~ NDZJH R2- SBC SEW SHN SPG T9H UQL WUQ ZY4 ~HD 7SC 7SP 7U5 8FD AFXIZ AGCQF AGRNS BNPGV JQ2 L7M L~C L~D SSH AALMO ABPIF ABPTK ABQIS EFJIC OTOTI |
| ID | FETCH-LOGICAL-c461t-d200a54e250ad40cc0ce23ca9ca186779659dbc6314fa2f210f0317849b5e5b83 |
| ISICitedReferencesCount | 689 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000507854200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0021-9991 |
| IngestDate | Thu May 18 22:32:26 EDT 2023 Sun Jul 13 05:18:24 EDT 2025 Tue Nov 18 22:12:01 EST 2025 Sat Nov 29 03:10:26 EST 2025 Fri Feb 23 02:48:16 EST 2024 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | C |
| Keywords | Deep learning benchmarks Bad minima Inverse problems Partial differential equations Physics-informed neural networks Machine learning |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c461t-d200a54e250ad40cc0ce23ca9ca186779659dbc6314fa2f210f0317849b5e5b83 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 USDOE |
| ORCID | 0000-0002-8831-1000 0000000288311000 |
| OpenAccessLink | https://doi.org/10.1016/j.jcp.2019.109136 |
| PQID | 2353034604 |
| PQPubID | 2047462 |
| ParticipantIDs | osti_scitechconnect_1775904 proquest_journals_2353034604 crossref_primary_10_1016_j_jcp_2019_109136 crossref_citationtrail_10_1016_j_jcp_2019_109136 elsevier_sciencedirect_doi_10_1016_j_jcp_2019_109136 |
| PublicationCentury | 2000 |
| PublicationDate | 2020-03-01 2020-03-00 20200301 |
| PublicationDateYYYYMMDD | 2020-03-01 |
| PublicationDate_xml | – month: 03 year: 2020 text: 2020-03-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | Cambridge |
| PublicationPlace_xml | – name: Cambridge – name: United States |
| PublicationTitle | Journal of computational physics |
| PublicationYear | 2020 |
| Publisher | Elsevier Inc Elsevier Science Ltd Elsevier |
| Publisher_xml | – name: Elsevier Inc – name: Elsevier Science Ltd – name: Elsevier |
| References | Caudrey, Eilbeck, Gibbon (br0100) 1975; 25 Raissi, Wang, Triantafyllou, Karniadakis (br0310) 2019; 861 Yu (br0410) 2002 Haykin (br0150) 1999 Basdevant (br0030) 1986; 14 Sommerfeld (br0340) 1949 Bengio, Simard, Frasconi (br0060) 1994; 5 Raissi, Perdikaris, Karniadakis (br0270) 2018; 40 Raissi, Perdikaris, Karniadakis (br0280) 2019; 378 He, Zhang, Ren, Sun (br0160) 2016 LeCun, Bottou, Bengio, Haffner (br0180) 1998; 86 Raissi, Perdikaris, Karniadakis (br0290) 2017; 335 Rudy (br0330) 2017; 3 Kingma, Ba (br0170) 2017 Wazwaz (br0370) 2008; 13 Parish, Duraisamy (br0230) 2016; 305 Xu, Zhang, Luo, Xiao, Ma (br0390) 2019 Arpit (br0010) 2017 Whitham (br0380) 2011 Duraisamy, Zhang, Singh (br0130) 2015 Li, Li, Rong (br0200) 2013; 22 Owhadi (br0220) 2015; 13 Ruder (br0320) 2017 Raissi, Perdikaris, Karniadakis (br0300) 2017; 348 Dodd, Eilbeck, Gibbon, Morris (br0120) 1982 Baydin, Pearlmutter, Radul, Siskind (br0050) 2018; 18 Shen, Wang, Chen, Cheng (br0350) 2004; 20 Zhang, Duraisamy (br0420) 2015 Burgers (br0090) 1948 Raissi, Karniadakis (br0260) 2018; 357 Bateman (br0040) 1915; 43 Lagaris, Likas, Fotiadis (br0190) 1998; 9 Qian (br0240) January 2018; 272 Bertsekas (br0080) 1999 Rahaman (br0250) 2018 Wang (br0360) Berg, Nyström (br0070) 2019; 384 Dushkoff, Ptucha (br0140) 2016 Dodd (10.1016/j.jcp.2019.109136_br0120) 1982 Raissi (10.1016/j.jcp.2019.109136_br0300) 2017; 348 Dushkoff (10.1016/j.jcp.2019.109136_br0140) 2016 Kingma (10.1016/j.jcp.2019.109136_br0170) Bateman (10.1016/j.jcp.2019.109136_br0040) 1915; 43 Caudrey (10.1016/j.jcp.2019.109136_br0100) 1975; 25 He (10.1016/j.jcp.2019.109136_br0160) 2016 LeCun (10.1016/j.jcp.2019.109136_br0180) 1998; 86 Qian (10.1016/j.jcp.2019.109136_br0240) 2018; 272 Whitham (10.1016/j.jcp.2019.109136_br0380) 2011 Arpit (10.1016/j.jcp.2019.109136_br0010) Raissi (10.1016/j.jcp.2019.109136_br0310) 2019; 861 Haykin (10.1016/j.jcp.2019.109136_br0150) 1999 Owhadi (10.1016/j.jcp.2019.109136_br0220) 2015; 13 Basdevant (10.1016/j.jcp.2019.109136_br0030) 1986; 14 Baydin (10.1016/j.jcp.2019.109136_br0050) 2018; 18 Bengio (10.1016/j.jcp.2019.109136_br0060) 1994; 5 Shen (10.1016/j.jcp.2019.109136_br0350) 2004; 20 Wazwaz (10.1016/j.jcp.2019.109136_br0370) 2008; 13 Rudy (10.1016/j.jcp.2019.109136_br0330) 2017; 3 Burgers (10.1016/j.jcp.2019.109136_br0090) 1948 Li (10.1016/j.jcp.2019.109136_br0200) 2013; 22 Bertsekas (10.1016/j.jcp.2019.109136_br0080) 1999 Berg (10.1016/j.jcp.2019.109136_br0070) 2019; 384 Raissi (10.1016/j.jcp.2019.109136_br0280) 2019; 378 Rahaman (10.1016/j.jcp.2019.109136_br0250) Raissi (10.1016/j.jcp.2019.109136_br0270) 2018; 40 Xu (10.1016/j.jcp.2019.109136_br0390) Zhang (10.1016/j.jcp.2019.109136_br0420) 2015 Ruder (10.1016/j.jcp.2019.109136_br0320) Lagaris (10.1016/j.jcp.2019.109136_br0190) 1998; 9 Raissi (10.1016/j.jcp.2019.109136_br0290) 2017; 335 Duraisamy (10.1016/j.jcp.2019.109136_br0130) 2015 Wang (10.1016/j.jcp.2019.109136_br0360) Yu (10.1016/j.jcp.2019.109136_br0410) 2002 Parish (10.1016/j.jcp.2019.109136_br0230) 2016; 305 Raissi (10.1016/j.jcp.2019.109136_br0260) 2018; 357 Sommerfeld (10.1016/j.jcp.2019.109136_br0340) 1949 |
| References_xml | – volume: 272 start-page: 204 year: January 2018 end-page: 212 ident: br0240 article-title: Adaptive activation functions in convolutional neural networks publication-title: Neurocomputing – volume: 13 start-page: 812 year: 2015 end-page: 828 ident: br0220 article-title: Bayesian numerical homogenization publication-title: Multiscale Model. Simul. – year: 2017 ident: br0320 article-title: An overview of gradient descent optimization algorithms – volume: 43 start-page: 163 year: 1915 end-page: 170 ident: br0040 article-title: Some recent researches on the motion of fluids publication-title: Mon. Weather Rev. – volume: 18 start-page: 1 year: 2018 end-page: 43 ident: br0050 article-title: Automatic differentiation in machine learning: a survey publication-title: J. Mach. Learn. Res. – year: 2015 ident: br0420 article-title: Machine Learning Methods for Data-Driven Turbulence Modeling – start-page: 171 year: 1948 end-page: 199 ident: br0090 article-title: A mathematical model illustrating the theory of turbulence publication-title: Advances in Applied Mechanics, vol. 1 – volume: 22 start-page: 531 year: 2013 end-page: 539 ident: br0200 article-title: The extreme learning machine learning algorithm with tunable activation function publication-title: Neural Comput. Appl. – volume: 305 start-page: 758 year: 2016 end-page: 774 ident: br0230 article-title: A paradigm for data-driven predictive modeling using field inversion and machine learning publication-title: J. Comput. Phys. – year: 2015 ident: br0130 article-title: New Approaches in Turbulence and Transition Modeling Using Data-Driven Techniques – volume: 378 start-page: 686 year: 2019 end-page: 707 ident: br0280 article-title: Physics-informed neural network: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations publication-title: J. Comput. Phys. – volume: 357 start-page: 125 year: 2018 end-page: 141 ident: br0260 article-title: Hidden physics models: machine learning of nonlinear partial differential equations publication-title: J. Comput. Phys. – volume: 9 start-page: 987 year: 1998 end-page: 1000 ident: br0190 article-title: Artificial neural network for solving ordinary and partial differential equations publication-title: IEEE Trans. Neural Netw. – volume: 384 start-page: 239 year: 2019 end-page: 252 ident: br0070 article-title: Data-driven discovery of PDEs in complex datasets publication-title: J. Comput. Phys. – year: 2018 ident: br0250 article-title: On the spectral bias of deep neural networks – volume: 40 start-page: A172 year: 2018 end-page: A198 ident: br0270 article-title: Numerical Gaussian processes for time-dependent and nonlinear partial differential equations publication-title: SIAM J. Sci. Comput. – year: 2019 ident: br0390 article-title: Frequency principle: Fourier analysis sheds light on deep neural networks – volume: 86 start-page: 2278 year: 1998 end-page: 2324 ident: br0180 article-title: Gradient-based learning applied to document recognition publication-title: Proc. IEEE – volume: 20 start-page: 85 year: 2004 end-page: 104 ident: br0350 article-title: A new multi-output neural model with tunable activation function and its applications publication-title: Neural Process. Lett. – year: 2002 ident: br0410 article-title: An adaptive activation function for multilayer feedforward neural networks publication-title: 2002 IEEE Region 10 Conference on Computers, Communications, Control and Power Engineering. TENCOM '02. Proceedings – year: 1999 ident: br0150 article-title: Neural Networks: A Comprehensive Foundation – year: 1949 ident: br0340 article-title: Partial Differential Equations in Physics – year: 1999 ident: br0080 article-title: Nonlinear Programming – volume: 3 year: 2017 ident: br0330 article-title: Data-driven discovery of partial differential equations publication-title: Sci. Adv. – year: 2017 ident: br0010 article-title: A closer look at memorization in deep networks – volume: 13 start-page: 889 year: 2008 end-page: 901 ident: br0370 article-title: New traveling wave solutions to the Boussinesq and the Klein-Gordon equations publication-title: Commun. Nonlinear Sci. Numer. Simul. – start-page: 1 year: 2016 end-page: 5(5) ident: br0140 article-title: Adaptive activation functions for deep networks publication-title: Electronic Imaging, Computational Imaging XIV – volume: 25 start-page: 497 year: 1975 end-page: 511 ident: br0100 article-title: The sine-Gordon equation as a model classical field theory publication-title: Nuovo Cimento – year: 1982 ident: br0120 article-title: Solitons and Nonlinear Wave Equations – volume: 348 start-page: 683 year: 2017 end-page: 693 ident: br0300 article-title: Machine learning of linear differential equations using Gaussian processes publication-title: J. Comput. Phys. – start-page: 630 year: 2016 end-page: 645 ident: br0160 article-title: Identity mappings in deep residual networks publication-title: European Conference on Computer Vision – volume: 14 start-page: 23 year: 1986 end-page: 41 ident: br0030 article-title: Spectral and finite difference solution of the Burgers equation publication-title: Comput. Fluids – volume: 5 start-page: 157 year: 1994 end-page: 166 ident: br0060 article-title: Learning long-term dependencies with gradient descent is difficult publication-title: IEEE Trans. Neural Netw. – volume: 861 start-page: 119 year: 2019 end-page: 137 ident: br0310 article-title: Deep learning of vortex-induced vibrations publication-title: J. Fluid Mech. – year: 2017 ident: br0170 article-title: ADAM: a method for stochastic optimization – volume: 335 start-page: 736 year: 2017 end-page: 746 ident: br0290 article-title: Inferring solutions of differential equations using noisy multi-fidelity data publication-title: J. Comput. Phys. – year: 2011 ident: br0380 article-title: Linear and Nonlinear Waves, vol. 42 – ident: br0360 article-title: A comprehensive physics-informed machine learning framework for predictive turbulence modeling – start-page: 171 year: 1948 ident: 10.1016/j.jcp.2019.109136_br0090 article-title: A mathematical model illustrating the theory of turbulence doi: 10.1016/S0065-2156(08)70100-5 – year: 1949 ident: 10.1016/j.jcp.2019.109136_br0340 – ident: 10.1016/j.jcp.2019.109136_br0170 – volume: 40 start-page: A172 year: 2018 ident: 10.1016/j.jcp.2019.109136_br0270 article-title: Numerical Gaussian processes for time-dependent and nonlinear partial differential equations publication-title: SIAM J. Sci. Comput. doi: 10.1137/17M1120762 – volume: 9 start-page: 987 issue: 5 year: 1998 ident: 10.1016/j.jcp.2019.109136_br0190 article-title: Artificial neural network for solving ordinary and partial differential equations publication-title: IEEE Trans. Neural Netw. doi: 10.1109/72.712178 – volume: 384 start-page: 239 year: 2019 ident: 10.1016/j.jcp.2019.109136_br0070 article-title: Data-driven discovery of PDEs in complex datasets publication-title: J. Comput. Phys. doi: 10.1016/j.jcp.2019.01.036 – volume: 357 start-page: 125 year: 2018 ident: 10.1016/j.jcp.2019.109136_br0260 article-title: Hidden physics models: machine learning of nonlinear partial differential equations publication-title: J. Comput. Phys. doi: 10.1016/j.jcp.2017.11.039 – volume: 43 start-page: 163 issue: 4 year: 1915 ident: 10.1016/j.jcp.2019.109136_br0040 article-title: Some recent researches on the motion of fluids publication-title: Mon. Weather Rev. doi: 10.1175/1520-0493(1915)43<163:SRROTM>2.0.CO;2 – ident: 10.1016/j.jcp.2019.109136_br0360 – volume: 13 start-page: 812 year: 2015 ident: 10.1016/j.jcp.2019.109136_br0220 article-title: Bayesian numerical homogenization publication-title: Multiscale Model. Simul. doi: 10.1137/140974596 – year: 2011 ident: 10.1016/j.jcp.2019.109136_br0380 – year: 1999 ident: 10.1016/j.jcp.2019.109136_br0150 – ident: 10.1016/j.jcp.2019.109136_br0390 – year: 1982 ident: 10.1016/j.jcp.2019.109136_br0120 – volume: 3 issue: 4 year: 2017 ident: 10.1016/j.jcp.2019.109136_br0330 article-title: Data-driven discovery of partial differential equations publication-title: Sci. Adv. doi: 10.1126/sciadv.1602614 – volume: 348 start-page: 683 year: 2017 ident: 10.1016/j.jcp.2019.109136_br0300 article-title: Machine learning of linear differential equations using Gaussian processes publication-title: J. Comput. Phys. doi: 10.1016/j.jcp.2017.07.050 – volume: 18 start-page: 1 year: 2018 ident: 10.1016/j.jcp.2019.109136_br0050 article-title: Automatic differentiation in machine learning: a survey publication-title: J. Mach. Learn. Res. – volume: 305 start-page: 758 year: 2016 ident: 10.1016/j.jcp.2019.109136_br0230 article-title: A paradigm for data-driven predictive modeling using field inversion and machine learning publication-title: J. Comput. Phys. doi: 10.1016/j.jcp.2015.11.012 – volume: 25 start-page: 497 year: 1975 ident: 10.1016/j.jcp.2019.109136_br0100 article-title: The sine-Gordon equation as a model classical field theory publication-title: Nuovo Cimento doi: 10.1007/BF02724733 – volume: 14 start-page: 23 year: 1986 ident: 10.1016/j.jcp.2019.109136_br0030 article-title: Spectral and finite difference solution of the Burgers equation publication-title: Comput. Fluids doi: 10.1016/0045-7930(86)90036-8 – volume: 861 start-page: 119 year: 2019 ident: 10.1016/j.jcp.2019.109136_br0310 article-title: Deep learning of vortex-induced vibrations publication-title: J. Fluid Mech. doi: 10.1017/jfm.2018.872 – ident: 10.1016/j.jcp.2019.109136_br0320 – start-page: 630 year: 2016 ident: 10.1016/j.jcp.2019.109136_br0160 article-title: Identity mappings in deep residual networks – volume: 378 start-page: 686 year: 2019 ident: 10.1016/j.jcp.2019.109136_br0280 article-title: Physics-informed neural network: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations publication-title: J. Comput. Phys. doi: 10.1016/j.jcp.2018.10.045 – volume: 335 start-page: 736 year: 2017 ident: 10.1016/j.jcp.2019.109136_br0290 article-title: Inferring solutions of differential equations using noisy multi-fidelity data publication-title: J. Comput. Phys. doi: 10.1016/j.jcp.2017.01.060 – volume: 20 start-page: 85 year: 2004 ident: 10.1016/j.jcp.2019.109136_br0350 article-title: A new multi-output neural model with tunable activation function and its applications publication-title: Neural Process. Lett. doi: 10.1007/s11063-004-0637-4 – year: 1999 ident: 10.1016/j.jcp.2019.109136_br0080 – start-page: 1 year: 2016 ident: 10.1016/j.jcp.2019.109136_br0140 article-title: Adaptive activation functions for deep networks – volume: 86 start-page: 2278 year: 1998 ident: 10.1016/j.jcp.2019.109136_br0180 article-title: Gradient-based learning applied to document recognition publication-title: Proc. IEEE doi: 10.1109/5.726791 – volume: 272 start-page: 204 issue: 10 year: 2018 ident: 10.1016/j.jcp.2019.109136_br0240 article-title: Adaptive activation functions in convolutional neural networks publication-title: Neurocomputing doi: 10.1016/j.neucom.2017.06.070 – volume: 5 start-page: 157 issue: 2 year: 1994 ident: 10.1016/j.jcp.2019.109136_br0060 article-title: Learning long-term dependencies with gradient descent is difficult publication-title: IEEE Trans. Neural Netw. doi: 10.1109/72.279181 – ident: 10.1016/j.jcp.2019.109136_br0010 – year: 2015 ident: 10.1016/j.jcp.2019.109136_br0130 – volume: 13 start-page: 889 year: 2008 ident: 10.1016/j.jcp.2019.109136_br0370 article-title: New traveling wave solutions to the Boussinesq and the Klein-Gordon equations publication-title: Commun. Nonlinear Sci. Numer. Simul. doi: 10.1016/j.cnsns.2006.08.005 – year: 2002 ident: 10.1016/j.jcp.2019.109136_br0410 article-title: An adaptive activation function for multilayer feedforward neural networks – ident: 10.1016/j.jcp.2019.109136_br0250 – volume: 22 start-page: 531 year: 2013 ident: 10.1016/j.jcp.2019.109136_br0200 article-title: The extreme learning machine learning algorithm with tunable activation function publication-title: Neural Comput. Appl. doi: 10.1007/s00521-012-0858-9 – year: 2015 ident: 10.1016/j.jcp.2019.109136_br0420 |
| SSID | ssj0008548 |
| Score | 2.7295659 |
| Snippet | •We employed adaptive activation functions in deep and physics-informed neural networks.•The proposed method is very simple and it is shown to accelerate... We employ adaptive activation functions for regression in deep and physics-informed neural networks (PINNs) to approximate smooth and discontinuous functions... |
| SourceID | osti proquest crossref elsevier |
| SourceType | Open Access Repository Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 109136 |
| SubjectTerms | Activation Algorithms Bad minima Burgers equation Computational physics Computer simulation Convergence Critical point Deep learning benchmarks Frequencies Helmholtz equations Inverse problems Klein-Gordon equation Machine learning Mathematical analysis Neural networks Nonlinear differential equations Nonlinear equations Optimization Parameter identification Partial differential equations Physics-informed neural networks Topology |
| Title | Adaptive activation functions accelerate convergence in deep and physics-informed neural networks |
| URI | https://dx.doi.org/10.1016/j.jcp.2019.109136 https://www.proquest.com/docview/2353034604 https://www.osti.gov/biblio/1775904 |
| Volume | 404 |
| WOSCitedRecordID | wos000507854200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1090-2716 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0008548 issn: 0021-9991 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lj9MwELZQlwMXlqfoPpAPnFilysNJnGO1dAUcVhwWqTfLcZxVKzZbNWVZ_j0zHiepClvBgUsUJbVVeT7PjJ3P3zD2zpSiyuNKB5EwYQDxWAcy0XWQYvaaxC5Ku2IT-eWlnM-LL_5zQevKCeRNI-_vi9V_NTU8A2Pj0dl_MHffKTyAezA6XMHscP0rw08rvXJ8IDyycOe5hBC9iPKmjYFAg_oQRDhfkxin48TaFckGONO1AWmqIj3AOm2Ohhjj7QP5rHH1Ibq9Rd9JT9DR1xvtSuFNb-xPffZh0nt6_UNfY0UWf0pouRje4JGxClPcYff-bHazvVEBq9KeqUW7Zz7Ub3tjpIcUVK1rYskBh0UYxDmdv-w8tKAKxR6K53_0_LQJsZwsDaqQRsXEKZ7uqGzTsifP0wIlZA9iuJEjdjD9NJt_7uO3TAXFb__vum_hjhW40_1D2czoFhz0b-Hd5SxXz9hTbxw-JZA8Z49s84Id-oUH9269fcl0hxk-YIb3mOEDZvgWZvii4YgZDpjhu5jhhBneYeYV-3oxuzr_GPjaG4ERWbQJKphSOhUWMmRdidCY0Ng4Mbow2kkgog5lVZosiUSt4zqOwhrCQy5FUaY2LWXymo2a28a-YdwkeZ1X8CIqpajrrIyE1ZmQNrVlKatozMJuDJXxwvRYH-Wb6hiISwXDrnDYFQ37mL3vm6xIlWXfj0VnGOXTSkoXFWBnX7NjNCI2QTllg7wzaOPRM2YnnW2Vn_ytipMUMkKRheJob-Nj9mSYICdstFl_t6fssbnbLNr1W4_HX1Apq8c |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Adaptive+activation+functions+accelerate+convergence+in+deep+and+physics-informed+neural+networks&rft.jtitle=Journal+of+computational+physics&rft.au=Jagtap%2C+Ameya+D.&rft.au=Kawaguchi%2C+Kenji&rft.au=Karniadakis%2C+George+Em&rft.date=2020-03-01&rft.pub=Elsevier&rft.issn=0021-9991&rft.eissn=1090-2716&rft.volume=404&rft.issue=C&rft_id=info:doi/10.1016%2Fj.jcp.2019.109136&rft.externalDocID=1775904 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0021-9991&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0021-9991&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0021-9991&client=summon |