Stability Analysis of Denoising Autoencoders Based on Dynamical Projection System
In this study, we give a stability analysis of denoising autoencoder(DAE) from the novel perspective of dynamical systems when the input density is defined as a distribution on a manifold. We demonstrate the connection between the corrupted distribution and the learned reconstruction function of a n...
Saved in:
| Published in: | IEEE transactions on knowledge and data engineering Vol. 33; no. 8; pp. 3155 - 3159 |
|---|---|
| Main Authors: | , |
| Format: | Journal Article |
| Language: | English |
| Published: |
New York
IEEE
01.08.2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Subjects: | |
| ISSN: | 1041-4347, 1558-2191 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | In this study, we give a stability analysis of denoising autoencoder(DAE) from the novel perspective of dynamical systems when the input density is defined as a distribution on a manifold. We demonstrate the connection between the corrupted distribution and the learned reconstruction function of a nonlinear DAE, which motivates the use of a dynamic projection system (DPS) associated with the learned reconstruction function. Utilizing the constructed DPS, we prove that the high-density region of the corrupted data distribution asymptotically converges to the data manifold. Then, we show that the region is the attracting stable equilibrium manifold of the DPS which is completely stable. These results serve a theoretical basis of the DAE in recognizing the high-density region of the highly corrupted data with large deviations through the DPS. The effectiveness of this analysis is verified by conducting experiments on several toy examples and real image datasets with various types of noise. |
|---|---|
| AbstractList | In this study, we give a stability analysis of denoising autoencoder(DAE) from the novel perspective of dynamical systems when the input density is defined as a distribution on a manifold. We demonstrate the connection between the corrupted distribution and the learned reconstruction function of a nonlinear DAE, which motivates the use of a dynamic projection system (DPS) associated with the learned reconstruction function. Utilizing the constructed DPS, we prove that the high-density region of the corrupted data distribution asymptotically converges to the data manifold. Then, we show that the region is the attracting stable equilibrium manifold of the DPS which is completely stable. These results serve a theoretical basis of the DAE in recognizing the high-density region of the highly corrupted data with large deviations through the DPS. The effectiveness of this analysis is verified by conducting experiments on several toy examples and real image datasets with various types of noise. |
| Author | Park, Saerom Lee, Jaewook |
| Author_xml | – sequence: 1 givenname: Saerom orcidid: 0000-0002-2687-7105 surname: Park fullname: Park, Saerom email: psr6275@sungshin.ac.kr organization: Department of Convergence Security Engineering, Sungshin Women's University, Seoul, Korea – sequence: 2 givenname: Jaewook orcidid: 0000-0001-5720-8337 surname: Lee fullname: Lee, Jaewook email: jaewook@snu.ac.kr organization: Department of Industrial Engineering, Seoul National University, Seoul, Korea |
| BookMark | eNp9kMtOwzAQRS1UJErhAxCbSKxTxnn4sSxteYhKgFrWkZNMkKvULra7yN-TqBULFsxmRqN7RnfuJRkZa5CQGwpTSkHeb14Xy2kCCUxToJBwfkbGNM9FnFBJR_0MGY2zNOMX5NL7LQAILuiYfKyDKnWrQxfNjGo7r31km2iBxmqvzVc0OwSLprI1Oh89KI91ZE206Iza6Uq10buzW6yC7pfrzgfcXZHzRrUer099Qj4fl5v5c7x6e3qZz1Zxlcg0xDLF3o6QTcIBSs64VKzGNANWp7mQNJcolSwlMKQlY8hKiqoGJlRdiiyp0wm5O97dO_t9QB-KrT24_gdfJHkmec65ZL2KH1WVs947bIpKBzXYDU7ptqBQDPkVQ37FkF9xyq8n6R9y7_ROue5f5vbIaET81Uua9SXSH4z1fMk |
| CODEN | ITKEEH |
| CitedBy_id | crossref_primary_10_1016_j_knosys_2025_114334 crossref_primary_10_1109_TKDE_2023_3332784 |
| Cites_doi | 10.1162/089976603321780317 10.1162/089976698300017467 10.1109/TAC.2004.829603 10.1109/TPAMI.2014.2362140 10.1080/14786440109462720 10.1109/81.983867 10.1109/TPAMI.2014.2318727 10.1126/science.290.5500.2323 10.1126/science.290.5500.2319 10.1007/BF02288916 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TKDE.2020.3010277 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005-present IEEE All-Society Periodicals Package (ASPP) 1998-Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science |
| EISSN | 1558-2191 |
| EndPage | 3159 |
| ExternalDocumentID | 10_1109_TKDE_2020_3010277 9144448 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Research Foundation of Korea grantid: NRF-2019R1A2C2002358 funderid: 10.13039/501100003725 – fundername: National Research Foundation of Korea grantid: 2018R1D1A1A02085851 funderid: 10.13039/501100003725 |
| GroupedDBID | -~X .DC 0R~ 29I 4.4 5GY 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACIWK AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ IEDLZ IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNS RXW TAE TN5 UHB AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c293t-93e43489f2700b7679a6de3406d3589159e9a9b906e1b66e6b1ead068adb842d3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 3 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000671775200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1041-4347 |
| IngestDate | Sun Nov 09 08:01:34 EST 2025 Tue Nov 18 22:35:36 EST 2025 Sat Nov 29 02:36:02 EST 2025 Wed Aug 27 02:26:38 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 8 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c293t-93e43489f2700b7679a6de3406d3589159e9a9b906e1b66e6b1ead068adb842d3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0001-5720-8337 0000-0002-2687-7105 |
| PQID | 2549757796 |
| PQPubID | 85438 |
| PageCount | 5 |
| ParticipantIDs | proquest_journals_2549757796 ieee_primary_9144448 crossref_citationtrail_10_1109_TKDE_2020_3010277 crossref_primary_10_1109_TKDE_2020_3010277 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-08-01 |
| PublicationDateYYYYMMDD | 2021-08-01 |
| PublicationDate_xml | – month: 08 year: 2021 text: 2021-08-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on knowledge and data engineering |
| PublicationTitleAbbrev | TKDE |
| PublicationYear | 2021 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | bojchevski (ref13) 2017 kurakin (ref21) 2017 ref15 ref11 lecun (ref18) 2010; 2 ref10 xiao (ref19) 2017 ref2 madry (ref22) 2018 zhang (ref17) 2007 ref1 ref16 ref7 krizhevsky (ref20) 2009 bracewell (ref14) 1999 ref4 ref3 maaten (ref6) 2008; 9 goodfellow (ref8) 2016 alain (ref9) 2014; 15 kingma (ref12) 2010 roweis (ref5) 2000; 290 |
| References_xml | – year: 2018 ident: ref22 article-title: Towards deep learning models resistant to adversarial attacks publication-title: Proc Int Conf Learn Representations – ident: ref7 doi: 10.1162/089976603321780317 – year: 2017 ident: ref21 article-title: Adversarial machine learning at scale publication-title: Proc Int Conf Learn Representations – ident: ref3 doi: 10.1162/089976698300017467 – year: 2017 ident: ref13 article-title: Learning by denoising part 2. connection between data distribution and denoising function – year: 2017 ident: ref19 article-title: Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms – volume: 15 start-page: 3563 year: 2014 ident: ref9 article-title: What regularized auto-encoders learn from the data-generating distribution publication-title: J Mach Learn Res – ident: ref15 doi: 10.1109/TAC.2004.829603 – ident: ref10 doi: 10.1109/TPAMI.2014.2362140 – year: 2016 ident: ref8 publication-title: Deep Learning – start-page: 1126 year: 2010 ident: ref12 article-title: Regularized estimation of image statistics by score matching publication-title: Proc Advances Neural Inf Process Syst – ident: ref1 doi: 10.1080/14786440109462720 – ident: ref16 doi: 10.1109/81.983867 – start-page: 1593 year: 2007 ident: ref17 article-title: Mlle: Modified locally linear embedding using multiple weights publication-title: Proc Advances Neural Inf Process Syst – volume: 9 start-page: 2579 year: 2008 ident: ref6 article-title: Visualizing data using t-SNE publication-title: J Mach Learn Res – ident: ref11 doi: 10.1109/TPAMI.2014.2318727 – year: 2009 ident: ref20 article-title: Learning multiple layers of features from tiny images – start-page: 74 year: 1999 ident: ref14 article-title: The sifting property publication-title: The Fourier Transform and Its Applications – volume: 290 start-page: 2323 year: 2000 ident: ref5 article-title: Nonlinear dimensionality reduction by locally linear embedding publication-title: Science doi: 10.1126/science.290.5500.2323 – ident: ref4 doi: 10.1126/science.290.5500.2319 – volume: 2 year: 2010 ident: ref18 article-title: MNIST handwritten digit database publication-title: AT&T Labs – ident: ref2 doi: 10.1007/BF02288916 |
| SSID | ssj0008781 |
| Score | 2.3645272 |
| Snippet | In this study, we give a stability analysis of denoising autoencoder(DAE) from the novel perspective of dynamical systems when the input density is defined as... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 3155 |
| SubjectTerms | autoencoders data manifold Data models Density Dynamic stability dynamical systems Image reconstruction Manifolds Noise measurement Noise reduction Nonlinear projection Perturbation methods Reconstruction Stability analysis |
| Title | Stability Analysis of Denoising Autoencoders Based on Dynamical Projection System |
| URI | https://ieeexplore.ieee.org/document/9144448 https://www.proquest.com/docview/2549757796 |
| Volume | 33 |
| WOSCitedRecordID | wos000671775200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2191 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0008781 issn: 1041-4347 databaseCode: RIE dateStart: 19890101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFH7M4UEPTjfF6ZQcPIl17dLmx3G6DUEZEybsVtYmhYG0snWC_70v6Q8URfBWaEJLviTvfXkv3wO48jmNqUgCo1NJHV_4ylkyFTkqiRQTXoKUwoq4PvHpVCwWctaAm_oujNbaJp_pW_NoY_kqi7fmqKwv0ftHOrEDO5yz4q5WvesKbguSIrtATkR9XkYwPVf254-jMTLBARJUo6DG-TcbZIuq_NiJrXmZtP73Y4dwULqRZFjgfgQNnbahVZVoIOWKbcP-F73BDjyja2mTYT9IJUZCsoSMdJqtzJkBGW7zzChbmuxmcocGTpEsJaOiaj1-b1ac2yCWpJA6P4aXyXh-_-CUNRWcGA177kiqcXSETEzAOeKMS4RGUzTripoCg4HUcikj6TLtRYxpFnk411wmlioS_kDRE2imWapPgUgjrhd7Po9Vgk4Xup6x8IUKEvQZ8B3tgluNchiXguOm7sVraImHK0MDTGiACUtgunBdd3kr1Db-atwxSNQNSxC60KugDMv1uAkNDeYB55Kd_d7rHPYGJlvFpvb1oJmvt_oCduP3fLVZX9qp9gnjAc7x |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFD54A_XBuzivefBJrGuXLpdHdY6JcyhM8K2sTQoDaWVugv_ec9JsKIrgW6EJLf2SnPMlp98HcBpLnnGVN0mnkgexik0wECYNTJ4aoaIcKYUTce3KXk89P-uHOTif_QtjrXXFZ_aCLt1ZvimzCW2V1TVm_0gn5mGRnLP831qzdVdJZ0mK_AJZEY-lP8OMQl3v37VukAs2kKKShpqU36KQs1X5sRa7ANNe_9-rbcCaTyTZZYX8JszZYgvWpyYNzM_ZLVj9oji4DY-YXLpy2A82lSNhZc5atiiHtGvALifjkrQtqb6ZXWGIM6wsWKvyrcfnPVQ7N4gmq8TOd-CpfdO_7gTeVSHIMLSPA80tfh2lczpyTqWQGsGxHAO74WQx2NRWD3SqQ2GjVAgr0ghHWyjUwKQqbhi-CwtFWdg9YJrk9bIolpnJMe3C5DNTsTLNHLMGvMdrEE6_cpJ5yXFyvnhJHPUIdULAJARM4oGpwdmsy2ult_FX421CYtbQg1CDwymUiZ-RbwkRYdmUUov933udwHKnf99Nure9uwNYaVDtiiv0O4SF8Whij2Apex8P30bHbth9AtbG0jo |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Stability+Analysis+of+Denoising+Autoencoders+Based+on+Dynamical+Projection+System&rft.jtitle=IEEE+transactions+on+knowledge+and+data+engineering&rft.au=Park%2C+Saerom&rft.au=Lee%2C+Jaewook&rft.date=2021-08-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=1041-4347&rft.eissn=1558-2191&rft.volume=33&rft.issue=8&rft.spage=3155&rft_id=info:doi/10.1109%2FTKDE.2020.3010277&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1041-4347&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1041-4347&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1041-4347&client=summon |