Multilayer Convolutional Sparse Modeling: Pursuit and Dictionary Learning
The recently proposed multilayer convolutional sparse coding (ML-CSC) model, consisting of a cascade of convolutional sparse layers, provides a new interpretation of convolutional neural networks (CNNs). Under this framework, the forward pass in a CNN is equivalent to a pursuit algorithm aiming to e...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on signal processing Jg. 66; H. 15; S. 4090 - 4104 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
IEEE
01.08.2018
|
| Schlagworte: | |
| ISSN: | 1053-587X |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | The recently proposed multilayer convolutional sparse coding (ML-CSC) model, consisting of a cascade of convolutional sparse layers, provides a new interpretation of convolutional neural networks (CNNs). Under this framework, the forward pass in a CNN is equivalent to a pursuit algorithm aiming to estimate the nested sparse representation vectors from a given input signal. Despite having served as a pivotal connection between CNNs and sparse modeling, a deeper understanding of the ML-CSC is still lacking. In this paper, we propose a sound pursuit algorithm for the ML-CSC model by adopting a projection approach. We provide new and improved bounds on the stability of the solution of such pursuit and we analyze different practical alternatives to implement this in practice. We show that the training of the filters is essential to allow for nontrivial signals in the model, and we derive an online algorithm to learn the dictionaries from real data, effectively resulting in cascaded sparse convolutional layers. Last, but not least, we demonstrate the applicability of the ML-CSC model for several applications in an unsupervised setting, providing competitive results. Our work represents a bridge between matrix factorization, sparse dictionary learning, and sparse autoencoders, and we analyze these connections in detail. |
|---|---|
| AbstractList | The recently proposed multilayer convolutional sparse coding (ML-CSC) model, consisting of a cascade of convolutional sparse layers, provides a new interpretation of convolutional neural networks (CNNs). Under this framework, the forward pass in a CNN is equivalent to a pursuit algorithm aiming to estimate the nested sparse representation vectors from a given input signal. Despite having served as a pivotal connection between CNNs and sparse modeling, a deeper understanding of the ML-CSC is still lacking. In this paper, we propose a sound pursuit algorithm for the ML-CSC model by adopting a projection approach. We provide new and improved bounds on the stability of the solution of such pursuit and we analyze different practical alternatives to implement this in practice. We show that the training of the filters is essential to allow for nontrivial signals in the model, and we derive an online algorithm to learn the dictionaries from real data, effectively resulting in cascaded sparse convolutional layers. Last, but not least, we demonstrate the applicability of the ML-CSC model for several applications in an unsupervised setting, providing competitive results. Our work represents a bridge between matrix factorization, sparse dictionary learning, and sparse autoencoders, and we analyze these connections in detail. |
| Author | Elad, Michael Romano, Yaniv Papyan, Vardan Sulam, Jeremias |
| Author_xml | – sequence: 1 givenname: Jeremias surname: Sulam fullname: Sulam, Jeremias email: jsulam@cs.technion.ac.il organization: Dept. of Comput. Sci., Technion - Israel Inst. of Technol., Haifa, Israel – sequence: 2 givenname: Vardan surname: Papyan fullname: Papyan, Vardan email: vardanp91@gmail.com organization: Dept. of Stat., Stanford Univ., Stanford, CA, USA – sequence: 3 givenname: Yaniv surname: Romano fullname: Romano, Yaniv email: yromano@tx.technion.ac.il organization: Dept. of Stat., Stanford Univ., Stanford, CA, USA – sequence: 4 givenname: Michael surname: Elad fullname: Elad, Michael email: elad@cs.technion.ac.il organization: Dept. of Comput. Sci., Technion - Israel Inst. of Technol., Haifa, Israel |
| BookMark | eNotjstKAzEUQLOoYFvdC27yAzPm5jFJ3Mn4Kkyx0AruSiZzRyIxU-Yh9O-tj9XZHA5nQWapS0jIFbAcgNmb3XaTcwYm50YWnBczMgemRKaMfjsni2H4YAyktMWcrNZTHEN0R-xp2aWvLk5j6JKLdHtw_YB03TUYQ3q_pZupH6YwUpcaeh_8r9YfaYWuTyfhgpy1Lg54-c8leX182JXPWfXytCrvqiyAVmNW26JopUarpQDkQmlvpGVetdbWILhvrK8ZePhZbNEgE4a1DXdOgRIgxJJc_3UDIu4Pffg8XeyN0FYqJb4B8NJLeg |
| CODEN | ITPRED |
| ContentType | Journal Article |
| DBID | 97E RIA RIE |
| DOI | 10.1109/TSP.2018.2846226 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EndPage | 4104 |
| ExternalDocumentID | 8379455 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: European Research Council – fundername: European Unions Seventh Framework Programme ERC grantid: 320649 |
| GroupedDBID | -~X .DC 0R~ 29I 4.4 5GY 6IK 85S 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACIWK ACNCT AENEX AGQYO AHBIQ AJQPL AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 EBS EJD F5P HZ~ IFIPE IPLJI JAVBF LAI MS~ O9- OCL P2P RIA RIE RNS TAE TN5 |
| ID | FETCH-LOGICAL-i175t-b966f47e97431e2357c8490c5f99b132cd9cb01c10014fe8e0380fd2aa5153133 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 74 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000436458100002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1053-587X |
| IngestDate | Wed Aug 27 02:48:28 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 15 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i175t-b966f47e97431e2357c8490c5f99b132cd9cb01c10014fe8e0380fd2aa5153133 |
| PageCount | 15 |
| ParticipantIDs | ieee_primary_8379455 |
| PublicationCentury | 2000 |
| PublicationDate | 2018-Aug.1,-1 |
| PublicationDateYYYYMMDD | 2018-08-01 |
| PublicationDate_xml | – month: 08 year: 2018 text: 2018-Aug.1,-1 day: 01 |
| PublicationDecade | 2010 |
| PublicationTitle | IEEE transactions on signal processing |
| PublicationTitleAbbrev | TSP |
| PublicationYear | 2018 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0014496 |
| Score | 2.5832756 |
| Snippet | The recently proposed multilayer convolutional sparse coding (ML-CSC) model, consisting of a cascade of convolutional sparse layers, provides a new... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 4090 |
| SubjectTerms | Adaptation models Convolution Convolutional codes convolutional neural networks Convolutional sparse coding Data models Dictionaries dictionary learning Machine learning multilayer pursuit Pursuit algorithms sparse convolutional filters |
| Title | Multilayer Convolutional Sparse Modeling: Pursuit and Dictionary Learning |
| URI | https://ieeexplore.ieee.org/document/8379455 |
| Volume | 66 |
| WOSCitedRecordID | wos000436458100002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) issn: 1053-587X databaseCode: RIE dateStart: 19910101 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://ieeexplore.ieee.org/ omitProxy: false ssIdentifier: ssj0014496 providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09a8MwEBVJ6NAO_UpLv9HQsUqUWLakbqVtaJcQSArZgqyPYihOcOxA_31PshsydOlmBAb7ZHzv6e69Q-heUaulZYo4ESeEcaqITKQh1KhIpZBwNVdh2AQfj8V8Lict9LDVwlhrQ_OZ7fnLUMs3S135o7I-kCnJ4riN2pwntVZrWzFgLMziArgQkVjw-W9Jksr-bDrxPVyiB7_iZOhtFHYGqYQ8Mjr63xMco8MGL-KneoNPUMvmp-hgx0Wwi96DiPZLAXjGz8t803xMcNd0BbTVYj_vzKvOH_GkKtZVVmKVG_ySBUmDKr5xY7L6eYY-Rq-z5zfSTEggGaT9kqRAVhzjVnocYL1zjRZMUh07KVPgmdpIndKB9kZLzFlhaSSoM0OlAMZEQE_PUSdf5vYCYUAeYqA5oFilgAUpAe-eDpwzIqZDaeQl6vpwLFa1CcaiicTV38vXaN9HvO6Uu0GdsqjsLdrTmzJbF3dh534AamOZIQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA5zCurB3-Jvc_BotqxN18SbTMeGcww2YbeRJqkUpBtdO_C_9yWtYwcv3kqg0L6Uvu_Le9_3EHqQ1ChhmCQxD9qEhVQS0RaaUC19GUHCVaF0wybC4ZBPp2JUQ49rLYwxxjWfmYa9dLV8PVeFPSprApkSLAi20HbAmEdLtda6ZsCYm8YFgMEnAQ-nv0VJKpqT8ch2cfEG_IzbnjVS2Bil4jJJ9_B_z3CEDirEiJ_LLT5GNZOeoP0NH8FT1Hcy2i8J8Bl35umq-pzgrvECiKvBduKZ1Z0_4VGRLYskxzLV-CVxogaZfePKZvXzDH10XyedHqlmJJAEEn9OIqArMQuNsEjAWO8axZmgKoiFiIBpKi1URFvKWi2x2HBDfU5j7UkJQMYHgnqO6uk8NRcIA_bgLRUCjpUSeJDk8O5RK441D6gntLhEpzYcs0VpgzGrInH19_I92u1N3gezQX_4do32bPTLvrkbVM-zwtyiHbXKk2V253bxB5O1nGg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Multilayer+Convolutional+Sparse+Modeling%3A+Pursuit+and+Dictionary+Learning&rft.jtitle=IEEE+transactions+on+signal+processing&rft.au=Sulam%2C+Jeremias&rft.au=Papyan%2C+Vardan&rft.au=Romano%2C+Yaniv&rft.au=Elad%2C+Michael&rft.date=2018-08-01&rft.pub=IEEE&rft.issn=1053-587X&rft.volume=66&rft.issue=15&rft.spage=4090&rft.epage=4104&rft_id=info:doi/10.1109%2FTSP.2018.2846226&rft.externalDocID=8379455 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1053-587X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1053-587X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1053-587X&client=summon |