Parsimonious physics-informed random projection neural networks for initial value problems of ODEs and index-1 DAEs
We present a numerical method based on random projections with Gaussian kernels and physics-informed neural networks for the numerical solution of initial value problems (IVPs) of nonlinear stiff ordinary differential equations (ODEs) and index-1 differential algebraic equations (DAEs), which may al...
Gespeichert in:
| Veröffentlicht in: | Chaos (Woodbury, N.Y.) Jg. 33; H. 4 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
United States
01.04.2023
|
| ISSN: | 1089-7682, 1089-7682 |
| Online-Zugang: | Weitere Angaben |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | We present a numerical method based on random projections with Gaussian kernels and physics-informed neural networks for the numerical solution of initial value problems (IVPs) of nonlinear stiff ordinary differential equations (ODEs) and index-1 differential algebraic equations (DAEs), which may also arise from spatial discretization of partial differential equations (PDEs). The internal weights are fixed to ones while the unknown weights between the hidden and output layer are computed with Newton's iterations using the Moore-Penrose pseudo-inverse for low to medium scale and sparse QR decomposition with L 2 regularization for medium- to large-scale systems. Building on previous works on random projections, we also prove its approximation accuracy. To deal with stiffness and sharp gradients, we propose an adaptive step-size scheme and address a continuation method for providing good initial guesses for Newton iterations. The "optimal" bounds of the uniform distribution from which the values of the shape parameters of the Gaussian kernels are sampled and the number of basis functions are "parsimoniously" chosen based on bias-variance trade-off decomposition. To assess the performance of the scheme in terms of both numerical approximation accuracy and computational cost, we used eight benchmark problems (three index-1 DAEs problems, and five stiff ODEs problems including the Hindmarsh-Rose neuronal model of chaotic dynamics and the Allen-Cahn phase-field PDE). The efficiency of the scheme was compared against two stiff ODEs/DAEs solvers, namely, ode15s and ode23t solvers of the MATLAB ODE suite as well as against deep learning as implemented in the DeepXDE library for scientific machine learning and physics-informed learning for the solution of the Lotka-Volterra ODEs included in the demos of the library. A software/toolbox in Matlab (that we call RanDiffNet) with demos is also provided. |
|---|---|
| AbstractList | We present a numerical method based on random projections with Gaussian kernels and physics-informed neural networks for the numerical solution of initial value problems (IVPs) of nonlinear stiff ordinary differential equations (ODEs) and index-1 differential algebraic equations (DAEs), which may also arise from spatial discretization of partial differential equations (PDEs). The internal weights are fixed to ones while the unknown weights between the hidden and output layer are computed with Newton's iterations using the Moore-Penrose pseudo-inverse for low to medium scale and sparse QR decomposition with L 2 regularization for medium- to large-scale systems. Building on previous works on random projections, we also prove its approximation accuracy. To deal with stiffness and sharp gradients, we propose an adaptive step-size scheme and address a continuation method for providing good initial guesses for Newton iterations. The "optimal" bounds of the uniform distribution from which the values of the shape parameters of the Gaussian kernels are sampled and the number of basis functions are "parsimoniously" chosen based on bias-variance trade-off decomposition. To assess the performance of the scheme in terms of both numerical approximation accuracy and computational cost, we used eight benchmark problems (three index-1 DAEs problems, and five stiff ODEs problems including the Hindmarsh-Rose neuronal model of chaotic dynamics and the Allen-Cahn phase-field PDE). The efficiency of the scheme was compared against two stiff ODEs/DAEs solvers, namely, ode15s and ode23t solvers of the MATLAB ODE suite as well as against deep learning as implemented in the DeepXDE library for scientific machine learning and physics-informed learning for the solution of the Lotka-Volterra ODEs included in the demos of the library. A software/toolbox in Matlab (that we call RanDiffNet) with demos is also provided.We present a numerical method based on random projections with Gaussian kernels and physics-informed neural networks for the numerical solution of initial value problems (IVPs) of nonlinear stiff ordinary differential equations (ODEs) and index-1 differential algebraic equations (DAEs), which may also arise from spatial discretization of partial differential equations (PDEs). The internal weights are fixed to ones while the unknown weights between the hidden and output layer are computed with Newton's iterations using the Moore-Penrose pseudo-inverse for low to medium scale and sparse QR decomposition with L 2 regularization for medium- to large-scale systems. Building on previous works on random projections, we also prove its approximation accuracy. To deal with stiffness and sharp gradients, we propose an adaptive step-size scheme and address a continuation method for providing good initial guesses for Newton iterations. The "optimal" bounds of the uniform distribution from which the values of the shape parameters of the Gaussian kernels are sampled and the number of basis functions are "parsimoniously" chosen based on bias-variance trade-off decomposition. To assess the performance of the scheme in terms of both numerical approximation accuracy and computational cost, we used eight benchmark problems (three index-1 DAEs problems, and five stiff ODEs problems including the Hindmarsh-Rose neuronal model of chaotic dynamics and the Allen-Cahn phase-field PDE). The efficiency of the scheme was compared against two stiff ODEs/DAEs solvers, namely, ode15s and ode23t solvers of the MATLAB ODE suite as well as against deep learning as implemented in the DeepXDE library for scientific machine learning and physics-informed learning for the solution of the Lotka-Volterra ODEs included in the demos of the library. A software/toolbox in Matlab (that we call RanDiffNet) with demos is also provided. We present a numerical method based on random projections with Gaussian kernels and physics-informed neural networks for the numerical solution of initial value problems (IVPs) of nonlinear stiff ordinary differential equations (ODEs) and index-1 differential algebraic equations (DAEs), which may also arise from spatial discretization of partial differential equations (PDEs). The internal weights are fixed to ones while the unknown weights between the hidden and output layer are computed with Newton's iterations using the Moore-Penrose pseudo-inverse for low to medium scale and sparse QR decomposition with L 2 regularization for medium- to large-scale systems. Building on previous works on random projections, we also prove its approximation accuracy. To deal with stiffness and sharp gradients, we propose an adaptive step-size scheme and address a continuation method for providing good initial guesses for Newton iterations. The "optimal" bounds of the uniform distribution from which the values of the shape parameters of the Gaussian kernels are sampled and the number of basis functions are "parsimoniously" chosen based on bias-variance trade-off decomposition. To assess the performance of the scheme in terms of both numerical approximation accuracy and computational cost, we used eight benchmark problems (three index-1 DAEs problems, and five stiff ODEs problems including the Hindmarsh-Rose neuronal model of chaotic dynamics and the Allen-Cahn phase-field PDE). The efficiency of the scheme was compared against two stiff ODEs/DAEs solvers, namely, ode15s and ode23t solvers of the MATLAB ODE suite as well as against deep learning as implemented in the DeepXDE library for scientific machine learning and physics-informed learning for the solution of the Lotka-Volterra ODEs included in the demos of the library. A software/toolbox in Matlab (that we call RanDiffNet) with demos is also provided. |
| Author | Fabiani, Gianluca Siettos, Constantinos Galaris, Evangelos Russo, Lucia |
| Author_xml | – sequence: 1 givenname: Gianluca orcidid: 0000-0002-9799-451X surname: Fabiani fullname: Fabiani, Gianluca organization: Scuola Superiore Meridionale, Largo San Marcellino 10, 80138 Napoli (NA), Italy – sequence: 2 givenname: Evangelos orcidid: 0009-0004-7481-1356 surname: Galaris fullname: Galaris, Evangelos organization: Dipartimento di Matematica e Applicazioni "Renato Caccioppoli," Università degli Studi di Napoli Federico II, Corso Umberto I 40, 80138 Napoli (NA), Italy – sequence: 3 givenname: Lucia orcidid: 0000-0003-4888-467X surname: Russo fullname: Russo, Lucia organization: Istituto di Scienze e Tecnologie per l'Energia e la Mobilità Sostenibili, Consiglio Nazionale delle Ricerche, Via Guglielmo Marconi 4, 80125 Napoli (NA), Italy – sequence: 4 givenname: Constantinos orcidid: 0000-0002-9568-3355 surname: Siettos fullname: Siettos, Constantinos organization: Dipartimento di Matematica e Applicazioni "Renato Caccioppoli," Università degli Studi di Napoli Federico II, Corso Umberto I 40, 80138 Napoli (NA), Italy |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/37097940$$D View this record in MEDLINE/PubMed |
| BookMark | eNpNkEtPwzAQhC0Eog848AeQj1xS1nYSJ8eqLQ-pUjnAuXKcjXBJ7GInQP89RhSJ06x2Po12dkJOrbNIyBWDGYNc3GYzYCIrQZyQMYOiTGRe8NN_84hMQtgBAOMiOycjIaGUZQpjEp6UD6Zz1rgh0P3rIRgdEmMb5zusqVe2dh3de7dD3RtnqcXBqzZK_-n8W6ARpMaa3sTlh2oH_IGrFrtAXUM3y1WgMSMiNX4ljC7nq3BBzhrVBrw86pS83K2eFw_JenP_uJivEy0K0SeiBs4UZ1CxBkqEQlcadJZqVTSKYTRklipRZbKpJWZKQJUi5zoXQkqWMz4lN7-58aL3AUO_7UzQ2LbKYmy75QXkICGFNKLXR3SoYu_t3ptO-cP271H8G4bda3A |
| CitedBy_id | crossref_primary_10_1016_j_apm_2025_116127 crossref_primary_10_1016_j_jcp_2024_113433 crossref_primary_10_1016_j_cma_2023_116578 crossref_primary_10_1016_j_matcom_2024_12_004 crossref_primary_10_1137_24M1639890 crossref_primary_10_1016_j_cma_2024_117584 crossref_primary_10_1016_j_compbiolchem_2024_108291 crossref_primary_10_1088_1402_4896_ad6219 crossref_primary_10_3390_sym16111490 crossref_primary_10_1103_l8dq_mphg crossref_primary_10_1016_j_jcp_2025_113906 crossref_primary_10_1016_j_chaos_2024_115215 crossref_primary_10_1137_24M1686991 crossref_primary_10_1098_rsta_2024_0221 crossref_primary_10_3389_fams_2024_1221051 crossref_primary_10_1016_j_eswa_2024_124678 crossref_primary_10_1137_23M1602991 crossref_primary_10_1109_ACCESS_2024_3457670 crossref_primary_10_1016_j_compbiolchem_2025_108380 crossref_primary_10_1016_j_cam_2025_116635 crossref_primary_10_1103_PhysRevE_111_055302 crossref_primary_10_1016_j_cma_2024_116819 crossref_primary_10_1016_j_nexres_2025_100623 crossref_primary_10_1371_journal_pcbi_1011916 crossref_primary_10_1038_s41467_024_48024_7 |
| ContentType | Journal Article |
| Copyright | 2023 Author(s). Published under an exclusive license by AIP Publishing. |
| Copyright_xml | – notice: 2023 Author(s). Published under an exclusive license by AIP Publishing. |
| DBID | NPM 7X8 |
| DOI | 10.1063/5.0135903 |
| DatabaseName | PubMed MEDLINE - Academic |
| DatabaseTitle | PubMed MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic PubMed |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| Discipline | Sciences (General) Physics |
| EISSN | 1089-7682 |
| ExternalDocumentID | 37097940 |
| Genre | Journal Article |
| GroupedDBID | --- -~X .DC 1UP 2-P 29B 4.4 53G 5VS AAAAW AABDS AAEUA AAPUP AAYIH ABJGX ABJNI ACBRY ACGFS ACLYJ ACNCT ACZLF ADCTM AEJMO AENEX AFATG AFHCQ AGKCL AGLKD AGMXG AGTJO AHSDT AJJCW AJQPL ALEPV ALMA_UNASSIGNED_HOLDINGS AQWKA ATXIE AWQPM BDMKI BPZLN DU5 EBS ESX F5P FDOHQ FFFMQ HAM M6X M71 M73 N9A NPM NPSNA O-B P2P RIP RNS RQS TAE WH7 7X8 AAGWI AMVHM |
| ID | FETCH-LOGICAL-c383t-3d021a210b1f09e08cbc0c54ca8fa1e210754a3b57fd7e5a30b4e22c633771612 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 24 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000972044100002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1089-7682 |
| IngestDate | Fri Jul 11 08:06:59 EDT 2025 Wed Feb 19 02:23:36 EST 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 4 |
| Language | English |
| License | 2023 Author(s). Published under an exclusive license by AIP Publishing. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c383t-3d021a210b1f09e08cbc0c54ca8fa1e210754a3b57fd7e5a30b4e22c633771612 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| ORCID | 0000-0002-9568-3355 0000-0002-9799-451X 0000-0003-4888-467X 0009-0004-7481-1356 |
| PMID | 37097940 |
| PQID | 2806070404 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_2806070404 pubmed_primary_37097940 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-Apr-01 20230401 |
| PublicationDateYYYYMMDD | 2023-04-01 |
| PublicationDate_xml | – month: 04 year: 2023 text: 2023-Apr-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | Chaos (Woodbury, N.Y.) |
| PublicationTitleAlternate | Chaos |
| PublicationYear | 2023 |
| SSID | ssj0001235 |
| Score | 2.5253184 |
| Snippet | We present a numerical method based on random projections with Gaussian kernels and physics-informed neural networks for the numerical solution of initial... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| Title | Parsimonious physics-informed random projection neural networks for initial value problems of ODEs and index-1 DAEs |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/37097940 https://www.proquest.com/docview/2806070404 |
| Volume | 33 |
| WOSCitedRecordID | wos000972044100002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV05T8MwFLaAgsQCtFzlkpEYYLDq2M41oYq2YoDSAVC3yvEhdSApuOL385y46oSExJIhp_Xes_2OL99D6MZoZowVCYlpoXyAoonMGCc2SqgWlilRE5i-P6XjcTad5pOQcHMBVrlaE-uFWlfK58h7vgII5imouF98Et81yldXQwuNTdTi4Mp4SFc6XbOF-_9AG4B9TsCtZitmoYT3PFknj_O6V9YvnmW9w4z2_zu2A7QXfEvcb4yhjTZM2UE7NcZTuQ5qh3ns8G0gm747RG4CkS3oq_RYWNzkORxp6FSNxrCT6eoDh3wN6BB7Akz4SNnAxx2GG_HcI5DgpGcONzj0qHG4svhlMHQY3oFrVkYS4UF_6I7Q22j4-vBIQiMGoiCAXRKuwROQEBwWkaW5oZkqFFWxUDKzMjJwIY2F5EWcWp2aWHJaCMOYSjhPIR6L2DHaKqvSnCLMYptbzmkuo0IoISXXUcFNRLXmmcxNF12vRDwDQ_fVC1kakMBsLeQuOmn0NFs0jBwzntIcFhZ69oenz9GubxnfoG8uUMvCNDeXaFt9L-fu66q2IDiOJ88_LiLRDg |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Parsimonious+physics-informed+random+projection+neural+networks+for+initial+value+problems+of+ODEs+and+index-1+DAEs&rft.jtitle=Chaos+%28Woodbury%2C+N.Y.%29&rft.au=Fabiani%2C+Gianluca&rft.au=Galaris%2C+Evangelos&rft.au=Russo%2C+Lucia&rft.au=Siettos%2C+Constantinos&rft.date=2023-04-01&rft.issn=1089-7682&rft.eissn=1089-7682&rft.volume=33&rft.issue=4&rft_id=info:doi/10.1063%2F5.0135903&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1089-7682&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1089-7682&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1089-7682&client=summon |