Convolutional Dictionary Learning in Hierarchical Networks
Filter banks are a popular tool for the analysis of piecewise smooth signals such as natural images. Motivated by the empirically observed properties of scale and detail coefficients of images in the wavelet domain, we propose a hierarchical deep generative model of piecewise smooth signals that is...
Saved in:
| Published in: | 2019 IEEE 8th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP) pp. 131 - 135 |
|---|---|
| Main Authors: | , , , |
| Format: | Conference Proceeding |
| Language: | English |
| Published: |
IEEE
01.12.2019
|
| Subjects: | |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Filter banks are a popular tool for the analysis of piecewise smooth signals such as natural images. Motivated by the empirically observed properties of scale and detail coefficients of images in the wavelet domain, we propose a hierarchical deep generative model of piecewise smooth signals that is a recursion across scales: the low pass scale coefficients at one layer are obtained by filtering the scale coefficients at the next layer, and adding a high pass detail innovation obtained by filtering a sparse vector. This recursion describes a linear dynamic system that is a non-Gaussian Markov process across scales and is closely related to multilayer-convolutional sparse coding (ML-CSC) generative model for deep networks, except that our model allows for deeper architectures, and combines sparse and non-sparse signal representations. We propose an alternating minimization algorithm for learning the filters in this hierarchical model given observations at layer zero, e.g., natural images. The algorithm alternates between a coefficient-estimation step and a filter update step. The coefficient update step performs sparse (detail) and smooth (scale) coding and, when unfolded, leads to a deep neural network. We use MNIST to demonstrate the representation capabilities of the model, and its derived features (coefficients) for classification. |
|---|---|
| AbstractList | Filter banks are a popular tool for the analysis of piecewise smooth signals such as natural images. Motivated by the empirically observed properties of scale and detail coefficients of images in the wavelet domain, we propose a hierarchical deep generative model of piecewise smooth signals that is a recursion across scales: the low pass scale coefficients at one layer are obtained by filtering the scale coefficients at the next layer, and adding a high pass detail innovation obtained by filtering a sparse vector. This recursion describes a linear dynamic system that is a non-Gaussian Markov process across scales and is closely related to multilayer-convolutional sparse coding (ML-CSC) generative model for deep networks, except that our model allows for deeper architectures, and combines sparse and non-sparse signal representations. We propose an alternating minimization algorithm for learning the filters in this hierarchical model given observations at layer zero, e.g., natural images. The algorithm alternates between a coefficient-estimation step and a filter update step. The coefficient update step performs sparse (detail) and smooth (scale) coding and, when unfolded, leads to a deep neural network. We use MNIST to demonstrate the representation capabilities of the model, and its derived features (coefficients) for classification. |
| Author | Zazo, Javier Tolooshams, Bahareh Ba, Demba Paulson, Harvard John A. |
| Author_xml | – sequence: 1 givenname: Javier surname: Zazo fullname: Zazo, Javier organization: School of Engineering and Applied Sciences Harvard University – sequence: 2 givenname: Bahareh surname: Tolooshams fullname: Tolooshams, Bahareh organization: School of Engineering and Applied Sciences Harvard University – sequence: 3 givenname: Demba surname: Ba fullname: Ba, Demba organization: School of Engineering and Applied Sciences Harvard University – sequence: 4 givenname: Harvard John A. surname: Paulson fullname: Paulson, Harvard John A. organization: School of Engineering and Applied Sciences Harvard University |
| BookMark | eNotj8tOwzAURI0EC1r4Ajb5gQRfP-KYXRQeRQotErCu_LgBi2AjN4D4ewJ0NbM4OppZkMOYIhJSAK0AqD7v2ruH9l7IWtUVo6ArTRkTgh6QBSjWgJRCy2Ny0aX4mcaPKaRoxuIyuL-Wv4seTY4hPhchFquA2WT3EtzMrHH6Svl1d0KOBjPu8HSfS_J0ffXYrcp-c3PbtX0ZGOVTaZ2n1EgQFDxDlHVDrXfGaQVKcgvK1x4YNh4F48oKPUiLciYHJ39pviRn_96AiNv3HN7medv9Hf4DVKlGQA |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/CAMSAP45676.2019.9022440 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 1728155495 9781728155494 |
| EndPage | 135 |
| ExternalDocumentID | 9022440 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IL CBEJK RIE RIL |
| ID | FETCH-LOGICAL-i203t-bcd00a51401d2ee5680bdcac971753b17d6d12e8de4237b49f5be52eefc5e5683 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 1 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000556233000025&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Thu Jun 29 18:38:42 EDT 2023 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i203t-bcd00a51401d2ee5680bdcac971753b17d6d12e8de4237b49f5be52eefc5e5683 |
| PageCount | 5 |
| ParticipantIDs | ieee_primary_9022440 |
| PublicationCentury | 2000 |
| PublicationDate | 2019-Dec. |
| PublicationDateYYYYMMDD | 2019-12-01 |
| PublicationDate_xml | – month: 12 year: 2019 text: 2019-Dec. |
| PublicationDecade | 2010 |
| PublicationTitle | 2019 IEEE 8th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP) |
| PublicationTitleAbbrev | CAMSAP |
| PublicationYear | 2019 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| Score | 1.7467786 |
| Snippet | Filter banks are a popular tool for the analysis of piecewise smooth signals such as natural images. Motivated by the empirically observed properties of scale... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 131 |
| SubjectTerms | Analytical models Convolution Convolutional codes Convolutional dictionary learning deep networks Dictionaries Encoding hierarchical models Machine learning sparse coding Wavelet analysis |
| Title | Convolutional Dictionary Learning in Hierarchical Networks |
| URI | https://ieeexplore.ieee.org/document/9022440 |
| WOSCitedRecordID | wos000556233000025&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB5q8eBJpRXf5ODRtOluNrvxJtXS07IHhd5KHhNZKFvpC_z33WTXiuDFW0gGQh4w8yXfNwPwwCKDjmecasUs5SgkVVHkqHA-PpA6FsKEYhNpnmezmSw68HjQwiBiIJ_hwDfDX75dmq1_KhtK73B4DdCP0lQ0Wq1vcg6Tw3ENfZ-LOiBIPfVgJAet-a-6KcFtTE7_N-EZ9H_0d6Q4eJZz6GDVg6d6cNdeFLUgL2VQJKjVF2lzpH6QsiLT0iuKQ4GTBckbjve6D--T17fxlLaVD2gZsXhDtbGMqcSDHxshJiJj2hplZOoTa-pRaoUdRZhZ9KwWzaVLNCa1pTOJt44voFstK7wEEmvruLIojdNcZ1K5GgByFmvOFTKTXUHPr3v-2SS3mLdLvv67-wZO_NY2fI5b6G5WW7yDY7PblOvVfTiRPRBAj3o |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB5KFfSk0opvc_Bo2nQ3-4g3qZaKdemhQm8lj4ksyFb6Av-9m-xaEbx4C8mEMElgZpLvmwG4YYFGy1NOlWSGcowFlUFgaWydfyBUGMfaF5tIsiydTsW4AbdbLgwievAZdlzT_-WbuV67p7KucAaHlwH6TsR5wCq21jc8h4luvwx-78elS5A48EFPdOoJvyqneMMxOPjfkofQ_mHgkfHWthxBA4sW3JWDm_qqyHfykHtOglx8kjpL6hvJCzLMHafYlzh5J1mF8l624XXwOOkPaV37gOYBC1dUacOYjFz4YwLEKE6ZMlpqkbjUmqqXmNj0AkwNOlyL4sJGCqNS0urISYfH0CzmBZ4ACZWxXBoU2iquUiFtGQJyFirOJTKdnkLL6T37qNJbzGqVz_7uvoa94eRlNBs9Zc_nsO-2uUJ3XEBztVjjJezqzSpfLq786XwBtmeSwQ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2019+IEEE+8th+International+Workshop+on+Computational+Advances+in+Multi-Sensor+Adaptive+Processing+%28CAMSAP%29&rft.atitle=Convolutional+Dictionary+Learning+in+Hierarchical+Networks&rft.au=Zazo%2C+Javier&rft.au=Tolooshams%2C+Bahareh&rft.au=Ba%2C+Demba&rft.au=Paulson%2C+Harvard+John+A.&rft.date=2019-12-01&rft.pub=IEEE&rft.spage=131&rft.epage=135&rft_id=info:doi/10.1109%2FCAMSAP45676.2019.9022440&rft.externalDocID=9022440 |