Scalable Online Convolutional Sparse Coding
Convolutional sparse coding (CSC) improves sparse coding by learning a shift-invariant dictionary from the data. However, most existing CSC algorithms operate in the batch mode and are computationally expensive. In this paper, we alleviate this problem by online learning. The key is a reformulation...
Uloženo v:
| Vydáno v: | IEEE transactions on image processing Ročník 27; číslo 10; s. 4850 - 4859 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
United States
IEEE
01.10.2018
|
| Témata: | |
| ISSN: | 1057-7149, 1941-0042, 1941-0042 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Convolutional sparse coding (CSC) improves sparse coding by learning a shift-invariant dictionary from the data. However, most existing CSC algorithms operate in the batch mode and are computationally expensive. In this paper, we alleviate this problem by online learning. The key is a reformulation of the CSC objective so that convolution can be handled easily in the frequency domain, and much smaller history matrices are needed. To solve the resultant optimization problem, we use the alternating direction method of multipliers (ADMMs), and its subproblems have efficient closed-form solutions. Theoretical analysis shows that the learned dictionary converges to a stationary point of the optimization problem. Extensive experiments are performed on both the standard CSC benchmark data sets and much larger data sets such as the ImageNet. Results show that the proposed algorithm outperforms the state-of-the-art batch and online CSC methods. It is more scalable, has faster convergence, and better reconstruction performance. |
|---|---|
| AbstractList | Convolutional sparse coding (CSC) improves sparse coding by learning a shift-invariant dictionary from the data. However, most existing CSC algorithms operate in the batch mode and are computationally expensive. In this paper, we alleviate this problem by online learning. The key is a reformulation of the CSC objective so that convolution can be handled easily in the frequency domain, and much smaller history matrices are needed. To solve the resultant optimization problem, we use the alternating direction method of multipliers (ADMMs), and its subproblems have efficient closed-form solutions. Theoretical analysis shows that the learned dictionary converges to a stationary point of the optimization problem. Extensive experiments are performed on both the standard CSC benchmark data sets and much larger data sets such as the ImageNet. Results show that the proposed algorithm outperforms the state-of-the-art batch and online CSC methods. It is more scalable, has faster convergence, and better reconstruction performance.Convolutional sparse coding (CSC) improves sparse coding by learning a shift-invariant dictionary from the data. However, most existing CSC algorithms operate in the batch mode and are computationally expensive. In this paper, we alleviate this problem by online learning. The key is a reformulation of the CSC objective so that convolution can be handled easily in the frequency domain, and much smaller history matrices are needed. To solve the resultant optimization problem, we use the alternating direction method of multipliers (ADMMs), and its subproblems have efficient closed-form solutions. Theoretical analysis shows that the learned dictionary converges to a stationary point of the optimization problem. Extensive experiments are performed on both the standard CSC benchmark data sets and much larger data sets such as the ImageNet. Results show that the proposed algorithm outperforms the state-of-the-art batch and online CSC methods. It is more scalable, has faster convergence, and better reconstruction performance. Convolutional sparse coding (CSC) improves sparse coding by learning a shift-invariant dictionary from the data. However, most existing CSC algorithms operate in the batch mode and are computationally expensive. In this paper, we alleviate this problem by online learning. The key is a reformulation of the CSC objective so that convolution can be handled easily in the frequency domain, and much smaller history matrices are needed. To solve the resultant optimization problem, we use the alternating direction method of multipliers (ADMMs), and its subproblems have efficient closed-form solutions. Theoretical analysis shows that the learned dictionary converges to a stationary point of the optimization problem. Extensive experiments are performed on both the standard CSC benchmark data sets and much larger data sets such as the ImageNet. Results show that the proposed algorithm outperforms the state-of-the-art batch and online CSC methods. It is more scalable, has faster convergence, and better reconstruction performance. |
| Author | Yaqing Wang Quanming Yao Ni, Lionel M. Kwok, James T. |
| Author_xml | – sequence: 1 surname: Yaqing Wang fullname: Yaqing Wang email: ywangcy@ust.hk organization: Dept. of Comput. Sci. & Eng., Hong Kong Univ. of Sci. & Technol. Univ., Hong Kong, China – sequence: 2 surname: Quanming Yao fullname: Quanming Yao organization: Dept. of Comput. Sci. & Eng., Hong Kong Univ. of Sci. & Technol. Univ., Hong Kong, China – sequence: 3 givenname: James T. surname: Kwok fullname: Kwok, James T. email: jamesk@cse.ust.hk organization: Dept. of Comput. Sci. & Eng., Hong Kong Univ. of Sci. & Technol. Univ., Hong Kong, China – sequence: 4 givenname: Lionel M. surname: Ni fullname: Ni, Lionel M. organization: Dept. of Comput. & Inf. Sci., Univ. of Macau, Macau, China |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/29969396$$D View this record in MEDLINE/PubMed |
| BookMark | eNp9kM1Lw0AQxRep2A-9C4L0KEjqfmWTPUrxo1Co0HpeJpuprGw3NZsI_vcmtHrw4GmG4fce896YDEIVkJBLRmeMUX23WbzMOGX5jOeSs5SfkBHTkiWUSj7odppmScakHpJxjO-UMpkydUaGXGulhVYjcru24KHwOF0F7wJO51X4rHzbuCqAn673UMf-WLrwdk5Ot-AjXhznhLw-Pmzmz8ly9bSY3y8TK5hukrJAKIUoc5bqDBQVLC8xlwBlITDjGUUBOssBLdWFtMIWUnXfZALoVhbAxYTcHHz3dfXRYmzMzkWL3kPAqo2GUyV5yro0HXp9RNtih6XZ124H9Zf5CdgB6gDYuoqxxq2xroE-XVOD84ZR0zdpuiZN36Q5NtkJ6R_hj_c_kquDxCHiL54LpVkqxTeohHxW |
| CODEN | IIPRE4 |
| CitedBy_id | crossref_primary_10_1109_TCI_2018_2840334 crossref_primary_10_1016_j_jvcir_2023_103782 crossref_primary_10_1016_j_measurement_2023_112643 crossref_primary_10_1109_LSP_2021_3135196 crossref_primary_10_1109_TCYB_2019_2931914 crossref_primary_10_1016_j_engappai_2020_103758 crossref_primary_10_1007_s11042_023_17584_z crossref_primary_10_1016_j_image_2025_117356 crossref_primary_10_1016_j_sigpro_2019_107379 crossref_primary_10_1007_s13369_023_07716_w crossref_primary_10_1016_j_image_2024_117213 crossref_primary_10_1109_TIP_2024_3451693 crossref_primary_10_1007_s11042_023_15972_z crossref_primary_10_1109_TNNLS_2021_3119278 crossref_primary_10_1007_s11042_022_12395_0 crossref_primary_10_1016_j_jappgeo_2024_105610 crossref_primary_10_1109_TIP_2022_3141251 crossref_primary_10_1109_TSP_2020_2964239 crossref_primary_10_1109_TCI_2023_3340612 crossref_primary_10_1109_TIP_2020_2980980 crossref_primary_10_1007_s12652_022_03822_z crossref_primary_10_1016_j_apm_2021_02_023 crossref_primary_10_1016_j_bspc_2024_106859 crossref_primary_10_1145_3584860 crossref_primary_10_1016_j_image_2019_05_018 |
| Cites_doi | 10.1109/TSP.2006.881199 10.1109/TPAMI.2017.2656884 10.1561/2400000003 10.1017/CBO9780511546921 10.1109/TMI.2016.2570123 10.1111/cgf.12819 10.1137/1.9780898718881 10.1109/ICIP.2017.8296573 10.1016/j.dsp.2016.04.012 10.1109/CVPR.2010.5539957 10.1109/ICCV.2009.5459452 10.1137/110836936 10.1109/ICIP.2017.8296555 10.1561/2200000016 10.1561/2200000018 10.1109/CVPR.2015.7299149 10.1109/TIP.2015.2495260 10.1109/TSP.2004.830991 10.1109/ICVGIP.2008.47 10.1109/TASLP.2016.2598305 10.1109/CVPR.2013.57 10.1007/s00041-008-9045-x 10.1109/ICCV.2015.212 |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION NPM 7X8 |
| DOI | 10.1109/TIP.2018.2842152 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef PubMed MEDLINE - Academic |
| DatabaseTitle | CrossRef PubMed MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic PubMed |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: RIE name: IEEE Xplore digital library url: https://ieeexplore.ieee.org/ sourceTypes: Publisher – sequence: 3 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences Engineering |
| EISSN | 1941-0042 |
| EndPage | 4859 |
| ExternalDocumentID | 29969396 10_1109_TIP_2018_2842152 8369154 |
| Genre | orig-research Journal Article |
| GrantInformation_xml | – fundername: Universidade de Macau grantid: SRG2015-00050-FST funderid: 10.13039/501100004733 – fundername: Research Grants Council, University Grants Committee grantid: 614513 funderid: 10.13039/501100002920 |
| GroupedDBID | --- -~X .DC 0R~ 29I 4.4 53G 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABFSI ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 E.L EBS EJD F5P HZ~ H~9 ICLAB IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P RIA RIE RNS TAE TN5 VH1 AAYXX CITATION AAYOK NPM PKN RIG Z5M 7X8 |
| ID | FETCH-LOGICAL-c319t-dbead33d81597a60318de84aadb3e7270e3a978aec09b4c3cb4669373a0f4ba23 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 30 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000437412500004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1057-7149 1941-0042 |
| IngestDate | Sun Sep 28 05:22:17 EDT 2025 Wed Feb 19 02:40:50 EST 2025 Sat Nov 29 03:21:08 EST 2025 Tue Nov 18 21:00:59 EST 2025 Wed Aug 27 02:56:05 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 10 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c319t-dbead33d81597a60318de84aadb3e7270e3a978aec09b4c3cb4669373a0f4ba23 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| ORCID | 0000-0001-8944-8618 0000-0002-4828-8248 0000-0003-1457-1114 0000-0002-2325-6215 |
| PMID | 29969396 |
| PQID | 2064251014 |
| PQPubID | 23479 |
| PageCount | 10 |
| ParticipantIDs | proquest_miscellaneous_2064251014 crossref_primary_10_1109_TIP_2018_2842152 pubmed_primary_29969396 crossref_citationtrail_10_1109_TIP_2018_2842152 ieee_primary_8369154 |
| PublicationCentury | 2000 |
| PublicationDate | 2018-Oct. 2018-10-00 2018-Oct 20181001 |
| PublicationDateYYYYMMDD | 2018-10-01 |
| PublicationDate_xml | – month: 10 year: 2018 text: 2018-Oct. |
| PublicationDecade | 2010 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | IEEE transactions on image processing |
| PublicationTitleAbbrev | TIP |
| PublicationTitleAlternate | IEEE Trans Image Process |
| PublicationYear | 2018 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| References | ref35 ref13 ref34 ref12 ref15 ref14 jennings (ref32) 1992 hoffman (ref27) 2010 ref33 ref11 lee (ref2) 2007 ref1 ref38 ref16 ref19 šorel (ref17) 2016; 55 ref18 feng (ref28) 2013 khosla (ref36) 2011 bordes (ref26) 2005; 6 chew (ref31) 1995; 522 mallat (ref30) 1999 ref24 ref23 yang (ref4) 2009 ref25 mensch (ref21) 2016 ref22 krizhevsky (ref37) 2009 ref8 andilla (ref10) 2014 ref7 mairal (ref20) 2010; 11 pachitariu (ref9) 2013 ref3 ref6 ref5 shen (ref29) 2016 |
| References_xml | – start-page: 64 year: 2014 ident: ref10 article-title: Sparse space-time deconvolution for calcium image analysis publication-title: Proc Adv Neural Inf Process Syst – ident: ref1 doi: 10.1109/TSP.2006.881199 – volume: 6 start-page: 1579 year: 2005 ident: ref26 article-title: Fast kernel classifiers with online and active learning publication-title: J Mach Learn Res – ident: ref12 doi: 10.1109/TPAMI.2017.2656884 – year: 1999 ident: ref30 publication-title: A Wavelet Tour of Signal Processing – volume: 11 start-page: 19 year: 2010 ident: ref20 article-title: Online learning for matrix factorization and sparse coding publication-title: J Mach Learn Res – ident: ref33 doi: 10.1561/2400000003 – ident: ref18 doi: 10.1017/CBO9780511546921 – year: 1992 ident: ref32 publication-title: Matrix Computation – start-page: 622 year: 2016 ident: ref29 article-title: Online low-rank subspace clustering by basis dictionary pursuit publication-title: Proc Int Conf Mach Learn – ident: ref11 doi: 10.1109/TMI.2016.2570123 – ident: ref7 doi: 10.1111/cgf.12819 – start-page: 404 year: 2013 ident: ref28 article-title: Online robust PCA via stochastic optimization publication-title: Proc Adv Neural Inf Process Syst – ident: ref24 doi: 10.1137/1.9780898718881 – ident: ref23 doi: 10.1109/ICIP.2017.8296573 – volume: 55 start-page: 44 year: 2016 ident: ref17 article-title: Fast convolutional sparse coding using matrix inversion lemma publication-title: Digit Signal Process doi: 10.1016/j.dsp.2016.04.012 – ident: ref14 doi: 10.1109/CVPR.2010.5539957 – start-page: 1794 year: 2009 ident: ref4 article-title: Linear spatial pyramid matching using sparse coding for image classification publication-title: Proc Conf Comput Vis Pattern Recognit – start-page: 1737 year: 2016 ident: ref21 article-title: Dictionary learning for massive matrix factorization publication-title: Proc Int Conf Mach Learn – volume: 522 year: 1995 ident: ref31 publication-title: Waves and Fields in Inhomogeneous Media – ident: ref3 doi: 10.1109/ICCV.2009.5459452 – start-page: 801 year: 2007 ident: ref2 article-title: Efficient sparse coding algorithms publication-title: Proc Adv Neural Inf Process Syst – ident: ref34 doi: 10.1137/110836936 – start-page: 1745 year: 2013 ident: ref9 article-title: Extracting regions of interest from biological images with convolutional sparse block coding publication-title: Proc Adv Neural Inf Process Syst – ident: ref22 doi: 10.1109/ICIP.2017.8296555 – ident: ref15 doi: 10.1561/2200000016 – year: 2009 ident: ref37 article-title: Learning multiple layers of features from tiny images – ident: ref19 doi: 10.1561/2200000018 – ident: ref8 doi: 10.1109/CVPR.2015.7299149 – start-page: 1 year: 2011 ident: ref36 article-title: Novel dataset for fine-grained image categorization: Stanford dogs publication-title: Proc 1st Workshop Fine-Grained Vis Categorization – ident: ref16 doi: 10.1109/TIP.2015.2495260 – ident: ref25 doi: 10.1109/TSP.2004.830991 – ident: ref35 doi: 10.1109/ICVGIP.2008.47 – ident: ref13 doi: 10.1109/TASLP.2016.2598305 – ident: ref5 doi: 10.1109/CVPR.2013.57 – ident: ref38 doi: 10.1007/s00041-008-9045-x – ident: ref6 doi: 10.1109/ICCV.2015.212 – start-page: 856 year: 2010 ident: ref27 article-title: Online learning for latent Dirichlet allocation publication-title: Proc Adv Neural Inf Process Syst |
| SSID | ssj0014516 |
| Score | 2.4722636 |
| Snippet | Convolutional sparse coding (CSC) improves sparse coding by learning a shift-invariant dictionary from the data. However, most existing CSC algorithms operate... |
| SourceID | proquest pubmed crossref ieee |
| SourceType | Aggregation Database Index Database Enrichment Source Publisher |
| StartPage | 4850 |
| SubjectTerms | Convolution Convolutional codes convolutional sparse coding Dictionaries dictionary learning Frequency-domain analysis Image coding Online learning Optimization Sparse matrices |
| Title | Scalable Online Convolutional Sparse Coding |
| URI | https://ieeexplore.ieee.org/document/8369154 https://www.ncbi.nlm.nih.gov/pubmed/29969396 https://www.proquest.com/docview/2064251014 |
| Volume | 27 |
| WOSCitedRecordID | wos000437412500004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Xplore digital library customDbUrl: eissn: 1941-0042 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014516 issn: 1057-7149 databaseCode: RIE dateStart: 19920101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dS8MwED-24YM-ON38mB-jgi-C3bqlTdpHGQ59GQMn7K2kyRWE0Y59_f1e0q4oqOBTS0nS9nLX-10vuR_AvQwHPAgkcz2tPQpQuHKjVDM3QcaFxkQJVpBNiMkknM-jaQ0eq70wiGgXn2HPnNpcvs7V1vwq64eMR-Ty61AXghd7taqMgSGctZnNQLiCYP8-JelF_dnr1KzhCnv0KTY0rt9ckOVU-R1eWjczbv7vAU_guISTzlMx_6dQw6wFzRJaOqXhrltw9KXuYBvIouXC7JlyikqjzijPdqUO0mhvS4p2zUXj187gffw8G724JWuCq8icNq5OSDkY0yEBFSEtibTG0JdSJwwJrXjIJIWOEpUXJb5iKvE5J5DCpJf6iRyyc2hkeYaX4PB0mAjUA5SB9rlvD0L7OiWUoFOJHejvBRmrsqS4YbZYxDa08KKYRB8b0cel6DvwUPVYFuU0_mjbNhKu2pXC7cDdfq5iMgWT35AZ5ts1daZgynxjqM1FMYlVZ_K69JIRv_p50Gs4NLcuVundQGOz2uItHKjd5mO96pK-zcOu1bdPKzjQdQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dS8MwED_mB6gPTufX_Kzgi2C3uqRp-yjDseEcAyfsraTJFYTRDbf593tJu6Kggk8tJQnt5S73u15yP4AbGd4L35fM9bT2KEARyo1SzdwEmQg0JipgOdlEMBiE43E0rMBdeRYGEe3mM2yYW5vL11O1NL_KmiETEbn8NdjwOW95-WmtMmdgKGdtbtMP3ICA_yop6UXNUW9odnGFDVqMDZHrNydkWVV-B5jW0XSq_3vFPdgtAKXzkGvAPlQwq0G1AJdOYbrzGux8qTx4AGTTcmJOTTl5rVGnPc0-Ci2k0V5mFO-ah8azHcJr53HU7roFb4KryKAWrk5IPRjTIUGVQFoaaY0hl1InDAmveMgkBY8SlRclXDGVcCEIpjDppTyRLXYE69k0wxNwRNpKAtT3KH3NBbeXQHOdEk7QqcQ6NFeCjFVRVNxwW0xiG1x4UUyij43o40L0dbgte8zyghp_tD0wEi7bFcKtw_VqrmIyBpPhkBlOl3PqTOGUWWWozXE-iWVn8rv0kZE4_XnQK9jqjp77cb83eDqDbfMa-Z69c1hfvC_xAjbVx-Jt_n5pte4TQkPS1A |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Scalable+Online+Convolutional+Sparse+Coding&rft.jtitle=IEEE+transactions+on+image+processing&rft.au=Wang%2C+Yaqing&rft.au=Yao%2C+Quanming&rft.au=Kwok%2C+James+T&rft.au=Ni%2C+Lionel+M&rft.date=2018-10-01&rft.eissn=1941-0042&rft.volume=27&rft.issue=10&rft.spage=4850&rft_id=info:doi/10.1109%2FTIP.2018.2842152&rft_id=info%3Apmid%2F29969396&rft.externalDocID=29969396 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1057-7149&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1057-7149&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1057-7149&client=summon |