Partial Hard Thresholding
We study iterative algorithms for compressed sensing that have an "orthogonalization" step at each iteration to keep the residual orthogonal to the span of those columns of the measurement matrix that have been selected so far. To unify the design and analysis of such algorithms, we propos...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on information theory Jg. 63; H. 5; S. 3029 - 3038 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
New York
IEEE
01.05.2017
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Schlagworte: | |
| ISSN: | 0018-9448, 1557-9654 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | We study iterative algorithms for compressed sensing that have an "orthogonalization" step at each iteration to keep the residual orthogonal to the span of those columns of the measurement matrix that have been selected so far. To unify the design and analysis of such algorithms, we propose a novel partial hard-thresholding (PHT) operator that is similar to the hard thresholding operator but restricts the amount by which the support set can change in one iteration. Using the PHT operator and its properties, we provide a general framework to prove support recovery results for iterative algorithms employing this operator as well as those employing the hard-thresholding operator. Next, based on the PHT operator, we propose a novel family of algorithms. At one end of our family of algorithms lie well-known hard thresholding algorithms iterative thresholding with inversion and hard thresholding pursuit, whereas at the other end, we get a novel algorithm that we call orthogonal matching pursuit with replacement (OMPR). Like the classic greedy algorithm OMP, OMPR too adds exactly one coordinate to the support of the iterate at each iteration based on the correlation with the current residual. However, unlike OMP, OMPR also removes one coordinate from the support. This simple change allows us to prove that OMPR has the best known guarantees for sparse recovery in terms of the restricted isometry property (RIP), a condition on the measurement matrix. In contrast, OMP is known to have very weak performance guarantees under RIP. Finally, we show that most of the existing "orthogonalized" iterative algorithms, such as CoSaMP, subspace pursuit, OMP, can be expressed using the PHT operator. As a pleasing consequence of our novel and generic results for the PHT operator, we provide the tightest known RIP analysis of all of the above-mentioned iterative algorithms: CoSaMP, subspace pursuit, and OMP. |
|---|---|
| AbstractList | We study iterative algorithms for compressed sensing that have an “orthogonalization” step at each iteration to keep the residual orthogonal to the span of those columns of the measurement matrix that have been selected so far. To unify the design and analysis of such algorithms, we propose a novel partial hard-thresholding (PHT) operator that is similar to the hard thresholding operator but restricts the amount by which the support set can change in one iteration. Using the PHT operator and its properties, we provide a general framework to prove support recovery results for iterative algorithms employing this operator as well as those employing the hard-thresholding operator. Next, based on the PHT operator, we propose a novel family of algorithms. At one end of our family of algorithms lie well-known hard thresholding algorithms iterative thresholding with inversion and hard thresholding pursuit, whereas at the other end, we get a novel algorithm that we call orthogonal matching pursuit with replacement (OMPR). Like the classic greedy algorithm OMP, OMPR too adds exactly one coordinate to the support of the iterate at each iteration based on the correlation with the current residual. However, unlike OMP, OMPR also removes one coordinate from the support. This simple change allows us to prove that OMPR has the best known guarantees for sparse recovery in terms of the restricted isometry property (RIP), a condition on the measurement matrix. In contrast, OMP is known to have very weak performance guarantees under RIP. Finally, we show that most of the existing “orthogonalized” iterative algorithms, such as CoSaMP, subspace pursuit, OMP, can be expressed using the PHT operator. As a pleasing consequence of our novel and generic results for the PHT operator, we provide the tightest known RIP analysis of all of the above-mentioned iterative algorithms: CoSaMP, subspace pursuit, and OMP. |
| Author | Tewari, Ambuj Dhillon, Inderjit S. Jain, Prateek |
| Author_xml | – sequence: 1 givenname: Prateek surname: Jain fullname: Jain, Prateek organization: Microsoft Research India, Bangalure, India – sequence: 2 givenname: Ambuj orcidid: 0000-0001-6969-7844 surname: Tewari fullname: Tewari, Ambuj organization: Department of Statistics and Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA – sequence: 3 givenname: Inderjit S. surname: Dhillon fullname: Dhillon, Inderjit S. organization: Department of Computer Science, The University of Texas at Austin, Austin, TX, USA |
| BookMark | eNp9kE1Lw0AQhhepYFu9K14KnlNnv3ePUqwtFPQQz8smmdgtMam76cF_b0qLBw-ehoH3mZd5JmTUdi0SckthTinYx3ydzxlQPWfKKGPggoyplDqzSooRGQNQk1khzBWZpLQbViEpG5O7Nx_74JvZysdqlm8jpm3XVKH9uCaXtW8S3pznlLwvn_PFKtu8vqwXT5usZJb2mS1R6LpAAVBS462iDAxXsuIoi4pXoLmwKDkCpQVwW_DKo9YF2lrY2tZ8Sh5Od_ex-zpg6t2uO8R2qHTU2OE3oywbUnBKlbFLKWLt9jF8-vjtKLijADcIcEcB7ixgQNQfpAy970PX9tGH5j_w_gQGRPzt0cZozQz_ASzqZ8A |
| CODEN | IETTAW |
| CitedBy_id | crossref_primary_10_1016_j_jco_2020_101469 crossref_primary_10_3390_a12020036 crossref_primary_10_1109_TIT_2022_3188459 crossref_primary_10_3390_math11122674 crossref_primary_10_3390_s18082487 crossref_primary_10_1007_s00041_024_10131_w crossref_primary_10_1109_LSP_2024_3426353 |
| Cites_doi | 10.1023/A:1012470815092 10.1016/j.crma.2008.03.014 10.1109/ACSSC.1993.342465 10.1109/TIT.2009.2016006 10.1093/imaiai/iau005 10.1016/j.acha.2009.04.002 10.1007/s10208-012-9135-7 10.1016/j.acha.2011.04.005 10.1007/BF03549495 10.1016/j.acha.2008.07.002 10.1109/TIT.2011.2162263 10.1007/BF02678430 10.1109/TIT.2006.871582 10.1073/pnas.0909892106 10.1109/JSTSP.2009.2039176 10.1137/100806278 10.1002/cpa.20227 10.1109/TIT.2005.858979 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2017 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2017 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TIT.2017.2686880 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science |
| EISSN | 1557-9654 |
| EndPage | 3038 |
| ExternalDocumentID | 10_1109_TIT_2017_2686880 7887728 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: NSF (for ISD) and (for AT) grantid: CCF-1564000; DMS-1612549 |
| GroupedDBID | -~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABFSI ABQJQ ABVLG ACGFO ACGFS ACGOD ACIWK AENEX AETEA AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 E.L EBS EJD F5P HZ~ H~9 IAAWW IBMZZ ICLAB IDIHD IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNS RXW TAE TN5 VH1 VJK AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D RIG |
| ID | FETCH-LOGICAL-c291t-9ce47fbe400c18a961208365d3e5bd3d07349e53e011b039b3dae77be9f49f9f3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 16 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000399939200024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0018-9448 |
| IngestDate | Sun Jun 29 14:05:19 EDT 2025 Sat Nov 29 03:31:37 EST 2025 Tue Nov 18 22:33:19 EST 2025 Wed Aug 27 06:31:22 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 5 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c291t-9ce47fbe400c18a961208365d3e5bd3d07349e53e011b039b3dae77be9f49f9f3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0001-6969-7844 |
| PQID | 1891108692 |
| PQPubID | 36024 |
| PageCount | 10 |
| ParticipantIDs | crossref_primary_10_1109_TIT_2017_2686880 ieee_primary_7887728 crossref_citationtrail_10_1109_TIT_2017_2686880 proquest_journals_1891108692 |
| PublicationCentury | 2000 |
| PublicationDate | 2017-05-01 |
| PublicationDateYYYYMMDD | 2017-05-01 |
| PublicationDate_xml | – month: 05 year: 2017 text: 2017-05-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on information theory |
| PublicationTitleAbbrev | TIT |
| PublicationYear | 2017 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref12 ref15 jain (ref22) 2011; 24 ref14 ref20 ref21 zhang (ref17) 2008 ref2 maleki (ref16) 2009 ref1 ref19 ref18 ref8 rauhut (ref10) 2008; 7 ref7 ref9 mo (ref11) 2011 ref4 ref3 ref6 ref5 |
| References_xml | – ident: ref18 doi: 10.1023/A:1012470815092 – ident: ref5 doi: 10.1016/j.crma.2008.03.014 – ident: ref8 doi: 10.1109/ACSSC.1993.342465 – year: 2011 ident: ref11 article-title: Remarks on the restricted isometry property in orthogonal matching pursuit algorithm – ident: ref15 doi: 10.1109/TIT.2009.2016006 – ident: ref21 doi: 10.1093/imaiai/iau005 – ident: ref13 doi: 10.1016/j.acha.2009.04.002 – ident: ref20 doi: 10.1007/s10208-012-9135-7 – ident: ref6 doi: 10.1016/j.acha.2011.04.005 – volume: 7 start-page: 197 year: 2008 ident: ref10 article-title: On the impossibility of uniform sparse reconstruction using greedy methods publication-title: Sampling Theory Signal Image Process doi: 10.1007/BF03549495 – start-page: 1921 year: 2008 ident: ref17 article-title: Adaptive forward-backward greedy algorithm for sparse learning with linear model publication-title: Proc Adv Neural Inf Process Syst – ident: ref14 doi: 10.1016/j.acha.2008.07.002 – ident: ref12 doi: 10.1109/TIT.2011.2162263 – ident: ref9 doi: 10.1007/BF02678430 – ident: ref4 doi: 10.1109/TIT.2006.871582 – ident: ref7 doi: 10.1073/pnas.0909892106 – ident: ref1 doi: 10.1109/JSTSP.2009.2039176 – volume: 24 start-page: 1215 year: 2011 ident: ref22 article-title: Orthogonal matching pursuit with replacement publication-title: Proc Adv Neural Inf Process Syst – ident: ref2 doi: 10.1137/100806278 – ident: ref19 doi: 10.1002/cpa.20227 – start-page: 236 year: 2009 ident: ref16 article-title: Convergence analysis of iterative thresholding algorithms publication-title: Proc Allerton Conf Commun Control Comput – ident: ref3 doi: 10.1109/TIT.2005.858979 |
| SSID | ssj0014512 |
| Score | 2.3436227 |
| Snippet | We study iterative algorithms for compressed sensing that have an "orthogonalization" step at each iteration to keep the residual orthogonal to the span of... We study iterative algorithms for compressed sensing that have an “orthogonalization” step at each iteration to keep the residual orthogonal to the span of... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 3029 |
| SubjectTerms | Algorithms Classification algorithms Columns (structural) Compressed sensing Greedy algorithms Iterative algorithms iterative thresholding algorithms Matching pursuit algorithms Minimization Optimization techniques Recovery restricted isometry property Sparse matrices sparse recovery |
| Title | Partial Hard Thresholding |
| URI | https://ieeexplore.ieee.org/document/7887728 https://www.proquest.com/docview/1891108692 |
| Volume | 63 |
| WOSCitedRecordID | wos000399939200024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1557-9654 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014512 issn: 0018-9448 databaseCode: RIE dateStart: 19630101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB7a4kEPPlrFaJUcvAimTbJJducoYtFL6aFCb2EfExCklT78_e7mRUERvOWwu4Rv9svOZma-AbiTOjaakQmUZDpIMo2BRK2CTGrJJZNS6bBsNsGnU7FY4KwDD20tDBGVyWc0co9lLN-s9M79Khu7zDceiy50Oc-qWq02YpCkUaUMHlkC2ztHE5IMcTx_nbscLj6KM5EJJwC5dwSVPVV-fIjL02Vy8r_3OoXj2ov0Hyuzn0GHln04aTo0-DVh-3C0Jzc4AG_m9omd58L1_txacVMHn87hbfI8f3oJ6tYIgY4x2gaoKeGFIstAHQmJ1k9xMtOpYZQqw4wlboKUMrL0VSFDxYwkzhVhkWCBBbuA3nK1pEvwqYiKTBjGrSeUkBEolHUiwlS54mrFlQfjBq1c17rhrn3FR17eH0LMLb65wzev8fXgvp3xWWlm_DF24PBsx9VQejBsDJLXpNrkkUBXtJBhfPX7rGs4dGtX-YhD6G3XO7qBA_21fd-sb8v98g1RB7w- |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8NAEB1qFdSD1aoYrZqDF8G0STbJ7h5FLC3W0kOE3sJ-TECQVvrh73c3SUNBEbzlsMuGN_uys5mZNwB3QoVaEdSeFER5UaK4J7iSXiKUoIIIIZVfNJug4zGbTvmkAQ91LQwiFsln2LWPRSxfz9Xa_irr2cw3GrId2I2jKPTLaq06ZhDFQakNHhgKm1vHJijp8146TG0WF-2GCUuYlYDcOoSKrio_PsXF-dJv_e_NjuGo8iPdx9LwJ9DAWRtamx4NbkXZNhxuCQ6egjOxO8XMswF7NzV2XFbhpzN46z-nTwOvao7gqZAHK48rjGgu0XBQBUxw46lYoelYE4ylJtpQN-IYEzQElj7hkmiBlErkecRznpNzaM7mM7wAF_MgT5gm1PhCEWrGmTRuhB9LW14tqXSgt0ErU5VyuG1g8ZEVNwifZwbfzOKbVfg6cF_P-CxVM_4Ye2rxrMdVUDrQ2Rgkq2i1zALGbdlCwsPL32fdwv4gfR1lo-H45QoO7DpldmIHmqvFGq9hT32t3peLm2LvfANNVb-F |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Partial+Hard+Thresholding&rft.jtitle=IEEE+transactions+on+information+theory&rft.au=Jain%2C+Prateek&rft.au=Tewari%2C+Ambuj&rft.au=Dhillon%2C+Inderjit+S.&rft.date=2017-05-01&rft.pub=IEEE&rft.issn=0018-9448&rft.volume=63&rft.issue=5&rft.spage=3029&rft.epage=3038&rft_id=info:doi/10.1109%2FTIT.2017.2686880&rft.externalDocID=7887728 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9448&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9448&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9448&client=summon |