Stochastic privacy-preserving methods for nonconvex sparse learning
Sparse learning is essential in mining high-dimensional data. Iterative hard thresholding (IHT) methods are effective for optimizing nonconvex objectives for sparse learning. However, IHT methods are vulnerable to adversary attacks that infer sensitive data. Although pioneering works attempted to re...
Uloženo v:
| Vydáno v: | Information sciences Ročník 630; s. 567 - 585 |
|---|---|
| Hlavní autoři: | , , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Elsevier Inc
01.06.2023
|
| Témata: | |
| ISSN: | 0020-0255, 1872-6291 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Sparse learning is essential in mining high-dimensional data. Iterative hard thresholding (IHT) methods are effective for optimizing nonconvex objectives for sparse learning. However, IHT methods are vulnerable to adversary attacks that infer sensitive data. Although pioneering works attempted to relieve such vulnerability, they confront the issue of high computational cost for large-scale problems. We propose two differentially private stochastic IHT: one based on the stochastic gradient descent method (DP-SGD-HT) and the other based on the stochastically controlled stochastic gradient method (DP-SCSG-HT). The DP-SGD-HT method perturbs stochastic gradients with small Gaussian noise rather than full gradients, which are computationally expensive. As a result, computational complexity is reduced from O(nlog(n)) to a lower O(blog(n)), where n is the sample size and b is the mini-batch size used to compute stochastic gradients. The DP-SCSG-HT method further perturbs the stochastic gradients controlled by large-batch snapshot gradients to reduce stochastic gradient variance. We prove that both algorithms guarantee differential privacy and have linear convergence rates with estimation bias. A utility analysis examines the relationship between convergence rate and the level of perturbation, yielding the best-known utility bound for nonconvex sparse optimization. Extensive experiments show that our algorithms outperform existing methods. |
|---|---|
| AbstractList | Sparse learning is essential in mining high-dimensional data. Iterative hard thresholding (IHT) methods are effective for optimizing nonconvex objectives for sparse learning. However, IHT methods are vulnerable to adversary attacks that infer sensitive data. Although pioneering works attempted to relieve such vulnerability, they confront the issue of high computational cost for large-scale problems. We propose two differentially private stochastic IHT: one based on the stochastic gradient descent method (DP-SGD-HT) and the other based on the stochastically controlled stochastic gradient method (DP-SCSG-HT). The DP-SGD-HT method perturbs stochastic gradients with small Gaussian noise rather than full gradients, which are computationally expensive. As a result, computational complexity is reduced from O(nlog(n)) to a lower O(blog(n)), where n is the sample size and b is the mini-batch size used to compute stochastic gradients. The DP-SCSG-HT method further perturbs the stochastic gradients controlled by large-batch snapshot gradients to reduce stochastic gradient variance. We prove that both algorithms guarantee differential privacy and have linear convergence rates with estimation bias. A utility analysis examines the relationship between convergence rate and the level of perturbation, yielding the best-known utility bound for nonconvex sparse optimization. Extensive experiments show that our algorithms outperform existing methods. |
| Author | Tong, Qianqian Bi, Jinbo Liang, Guannan Pan, Miao Ding, Jiahao |
| Author_xml | – sequence: 1 givenname: Guannan surname: Liang fullname: Liang, Guannan email: guannan.liang@uconn.edu organization: University of Connecticut, United States – sequence: 2 givenname: Qianqian surname: Tong fullname: Tong, Qianqian email: qianqian.tong@uconn.edu organization: University of Connecticut, United States – sequence: 3 givenname: Jiahao surname: Ding fullname: Ding, Jiahao email: jding7@uh.edu organization: University of Houston, United States – sequence: 4 givenname: Miao surname: Pan fullname: Pan, Miao email: mpan2@uh.edu organization: University of Houston, United States – sequence: 5 givenname: Jinbo surname: Bi fullname: Bi, Jinbo email: jinbo.bi@uconn.edu organization: University of Connecticut, United States |
| BookMark | eNp9kEFOwzAQRS1UJNrCAdjlAgljJ7FjsUIVUKRKLIC15Thj6qq1KzuK6O1xVdas_ua_ma-3IDMfPBJyT6GiQPnDrnI-VQwYq0BWwNkVmdNOsJIzSWdkDsCgBNa2N2SR0g4AGsH5nKw-xmC2Oo3OFMfoJm1O5TFiwjg5_10ccNyGIRU2xCJ_NMFP-FOko44Jiz3q6HPrllxbvU9495dL8vXy_Llal5v317fV06Y0rJFj2VjNBLYoB9sKawXX2nCLBrGrZdcL4GhtbQ0ILinvjbSyMT0C7Rl2XU3rJaGXuyaGlCJalRcfdDwpCupsQe1UtqDOFhRIlS1k5vHCYB42OYwqGYfe4OAimlENwf1D_wLHRmmI |
| Cites_doi | 10.1109/MSP.2012.2211477 10.24963/ijcai.2019/519 10.1137/090756090 10.1016/j.cell.2018.02.010 10.1038/s41598-022-05539-7 10.1016/j.acha.2009.04.002 10.1109/ACCESS.2022.3151670 10.1016/j.ins.2022.05.128 10.1016/j.ins.2021.01.058 10.1016/j.ins.2022.04.030 10.1016/j.ins.2019.07.001 10.1109/78.258082 10.1609/aaai.v34i02.5519 10.1109/TIT.2017.2749330 |
| ContentType | Journal Article |
| Copyright | 2022 |
| Copyright_xml | – notice: 2022 |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.ins.2022.09.062 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Library & Information Science |
| EISSN | 1872-6291 |
| EndPage | 585 |
| ExternalDocumentID | 10_1016_j_ins_2022_09_062 S0020025522011161 |
| GroupedDBID | --K --M --Z -~X .DC .~1 0R~ 1B1 1OL 1RT 1~. 1~5 29I 4.4 457 4G. 5GY 5VS 7-5 71M 8P~ 9JN 9JO AAAKF AAAKG AABNK AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AARIN AAXUO AAYFN ABAOU ABBOA ABEFU ABFNM ABJNI ABMAC ABTAH ABUCO ABXDB ABYKQ ACAZW ACDAQ ACGFS ACNNM ACRLP ACZNC ADBBV ADEZE ADGUI ADJOM ADMUD ADTZH AEBSH AECPX AEKER AENEX AFFNX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHJVU AHZHX AIALX AIEXJ AIGVJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD APLSM ARUGR ASPBG AVWKF AXJTR AZFZN BJAXD BKOJK BLXMC CS3 DU5 EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN G-Q GBLVA GBOLZ HAMUX HLZ HVGLF HZ~ H~9 IHE J1W JJJVA KOM LG9 LY1 M41 MHUIS MO0 MS~ N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 R2- RIG ROL RPZ SBC SDF SDG SDP SDS SES SEW SPC SPCBC SSB SSD SST SSV SSW SSZ T5K TN5 TWZ UHS WH7 WUQ XPP YYP ZMT ZY4 ~02 ~G- 77I 9DU AATTM AAXKI AAYWO AAYXX ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO ADVLN AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP CITATION EFKBS ~HD |
| ID | FETCH-LOGICAL-c249t-4fa27e5e9df57ff76aac6fecee8398b706eff3fc076916bc9f94cbe01b2e88313 |
| ISICitedReferencesCount | 1 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000946723000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0020-0255 |
| IngestDate | Sat Nov 29 07:25:32 EST 2025 Fri Feb 23 02:37:44 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Differential privacy Stochastic algorithm Sparse learning |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c249t-4fa27e5e9df57ff76aac6fecee8398b706eff3fc076916bc9f94cbe01b2e88313 |
| PageCount | 19 |
| ParticipantIDs | crossref_primary_10_1016_j_ins_2022_09_062 elsevier_sciencedirect_doi_10_1016_j_ins_2022_09_062 |
| PublicationCentury | 2000 |
| PublicationDate | June 2023 2023-06-00 |
| PublicationDateYYYYMMDD | 2023-06-01 |
| PublicationDate_xml | – month: 06 year: 2023 text: June 2023 |
| PublicationDecade | 2020 |
| PublicationTitle | Information sciences |
| PublicationYear | 2023 |
| Publisher | Elsevier Inc |
| Publisher_xml | – name: Elsevier Inc |
| References | El Ouadrhiri, Abdelhadi (b0065) 2022; 10 X. Li, R. Arora, H. Liu, J. Haupt, and T. Zhao. Nonconvex sparse learning via stochastic optimization with progressive variance reduction. arXiv preprint arXiv:1605.02711, 2016a. Jiang, Niu, Ying, Wu, Luo (b0095) 2022; 602 Li, Li, Zhu, Huang (b0135) 2019; 503 Nguyen, Needell, Woolf (b0165) 2017; 63 A. Rényi et al. On measures of entropy and information. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics. The Regents of the University of California, 1961. P. Jain, A. Tewari, and P. Kar. On iterative hard thresholding methods for high-dimensional m-estimation. In Advances in Neural Information Processing Systems, pages 685–693, 2014. P. Zhou, X. Yuan, and J. Feng. Efficient stochastic gradient hard thresholding. In Advances in Neural Information Processing Systems, pages 1988–1997, 2018. M. Elibol, L. Lei, and M.I. Jordan. Variance reduction with sparse gradients. arXiv preprint arXiv:2001.09623, 2020. Guo, Ding, Wang, Jia (b0080) 2022 Kermany, Goldbaum, Cai, Valentim, Liang, Baxter, McKeown, Yang, Wu, Yan (b0110) 2018; 172 Pati, Rezaiifar, Krishnaprasad (b0170) 1993 Wang, Wang (b0205) 2021; 560 L. Wang and Q. Gu. A knowledge transfer framework for differentially private sparse learning. arXiv preprint arXiv:1909.06322, 2019b. Erlingsson, Pihur, Korolova (b0075) 2014 T. Li, A.K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith. Federated optimization in heterogeneous networks. arXiv preprint arXiv:1812.06127, 2018. K. Talwar, A.G. Thakurta, and L. Zhang. Nearly optimal private lasso. In Advances in Neural Information Processing Systems, pages 3025–3033, 2015. Wang, Xu (b0195) 2019 Mironov (b0155) 2017 M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G.S. Corrado, A. Davis, J. Dean, M. Devin, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467, 2016a. D. Kifer, A. Smith, and A. Thakurta. Private convex empirical risk minimization and high-dimensional regression. In Conference on Learning Theory, pages 25–1, 2012. Dwork, Rothblum, Vadhan (b0060) 2010 Dwork, Kenthapadi, McSherry, Mironov, Naor (b0050) 2006 Q. Tong, G. Liang, T. Zhu, and J. Bi. Federated nonconvex sparse learning. arXiv preprint arXiv:2101.00052, 2020. J. Near. Differential privacy at scale: Uber and berkeley collaboration. In Enigma 2018 (Enigma 2018), 2018. R. Johnson and T. Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In Advances in neural information processing systems, pages 315–323, 2013. D. Wang, M. Ye, and J. Xu. Differentially private empirical risk minimization revisited: Faster and more general. In Advances in Neural Information Processing Systems, pages 2722–2731, 2017. Wu, Li, Kumar, Chaudhuri, Jha, Naughton (b0230) 2017 Kasiviswanathan, Lee, Nissim, Raskhodnikova, Smith (b0105) 2011; 40 A.E.C. Cloud. Amazon web services. Retrieved November, 9 (2011): 2011, 2011. L. Lei and M. Jordan. Less than a single pass: Stochastically controlled stochastic gradient. In Artificial Intelligence and Statistics, pages 148–156, 2017. Deng (b0045) 2012; 29 Dwork, McSherry, Nissim, Smith (b0055) 2006 L. Wang and Q. Gu. Differentially private iterative gradient hard thresholding for sparse learning. In 28th International Joint Conference on Artificial Intelligence, 2019a. Y.-X. Wang, B. Balle, and S. Kasiviswanathan. Subsampled r⧹)ényi differential privacy and analytical moments accountant. arXiv preprint arXiv:1808.00087, 2018. Zhu, Fang, Guo, Niu, Cao, Yue, Liu (b0240) 2009 Abadi, Chu, Goodfellow, McMahan, Mironov, Talwar, Zhang (b0010) 2016 Adnan, Kalra, Cresswell, Taylor, Tizhoosh (b0015) 2022; 12 L. Wang, B. Jayaraman, D. Evans, and Q. Gu. Efficient privacy-preserving nonconvex optimization. arXiv preprint arXiv:1910.13659, 2019. Bahmani, Raj, Boufounos (b0020) 2013; 14 Bassily, Smith, Thakurta (b0025) 2014 Truex, Liu, Gursoy, Yu, Wei (b0190) 2019 X. Li, T. Zhao, R. Arora, H. Liu, and J. Haupt. Stochastic variance reduced optimization for nonconvex sparse learning. In International Conference on Machine Learning, pages 917–925, 2016b. G. Liang, Q. Tong, C.J. Zhu, and J. Bi. An effective hard thresholding method based on stochastic variance reduction for nonconvex sparse learning. In AAAI, pages 1585–1592, 2020. Chaudhuri, Monteleoni, Sarwate (b0035) 2011; 12 Blumensath, Davies (b0030) 2009; 27 R.B. Harikandeh, M.O. Ahmed, A. Virani, M. Schmidt, J. Konečný, and S. Sallinen. Stopwasting my gradients: Practical svrg. In Advances in Neural Information Processing Systems, pages 2251–2259, 2015. Mallat, Zhang (b0150) 1993; 41 10.1016/j.ins.2022.09.062_b0120 El Ouadrhiri (10.1016/j.ins.2022.09.062_b0065) 2022; 10 10.1016/j.ins.2022.09.062_b0185 10.1016/j.ins.2022.09.062_b0100 10.1016/j.ins.2022.09.062_b0220 10.1016/j.ins.2022.09.062_b0145 10.1016/j.ins.2022.09.062_b0200 10.1016/j.ins.2022.09.062_b0005 Jiang (10.1016/j.ins.2022.09.062_b0095) 2022; 602 10.1016/j.ins.2022.09.062_b0225 10.1016/j.ins.2022.09.062_b0125 Kermany (10.1016/j.ins.2022.09.062_b0110) 2018; 172 10.1016/j.ins.2022.09.062_b0180 10.1016/j.ins.2022.09.062_b0160 10.1016/j.ins.2022.09.062_b0085 10.1016/j.ins.2022.09.062_b0140 10.1016/j.ins.2022.09.062_b0040 Blumensath (10.1016/j.ins.2022.09.062_b0030) 2009; 27 Chaudhuri (10.1016/j.ins.2022.09.062_b0035) 2011; 12 Mironov (10.1016/j.ins.2022.09.062_b0155) 2017 Truex (10.1016/j.ins.2022.09.062_b0190) 2019 Wang (10.1016/j.ins.2022.09.062_b0205) 2021; 560 Abadi (10.1016/j.ins.2022.09.062_b0010) 2016 Deng (10.1016/j.ins.2022.09.062_b0045) 2012; 29 Bassily (10.1016/j.ins.2022.09.062_b0025) 2014 Dwork (10.1016/j.ins.2022.09.062_b0050) 2006 Wu (10.1016/j.ins.2022.09.062_b0230) 2017 10.1016/j.ins.2022.09.062_b0215 Dwork (10.1016/j.ins.2022.09.062_b0055) 2006 10.1016/j.ins.2022.09.062_b0175 10.1016/j.ins.2022.09.062_b0130 10.1016/j.ins.2022.09.062_b0210 10.1016/j.ins.2022.09.062_b0115 Guo (10.1016/j.ins.2022.09.062_b0080) 2022 Li (10.1016/j.ins.2022.09.062_b0135) 2019; 503 10.1016/j.ins.2022.09.062_b0235 10.1016/j.ins.2022.09.062_b0090 10.1016/j.ins.2022.09.062_b0070 Mallat (10.1016/j.ins.2022.09.062_b0150) 1993; 41 Dwork (10.1016/j.ins.2022.09.062_b0060) 2010 Bahmani (10.1016/j.ins.2022.09.062_b0020) 2013; 14 Zhu (10.1016/j.ins.2022.09.062_b0240) 2009 Adnan (10.1016/j.ins.2022.09.062_b0015) 2022; 12 Kasiviswanathan (10.1016/j.ins.2022.09.062_b0105) 2011; 40 Wang (10.1016/j.ins.2022.09.062_b0195) 2019 Nguyen (10.1016/j.ins.2022.09.062_b0165) 2017; 63 Pati (10.1016/j.ins.2022.09.062_b0170) 1993 Erlingsson (10.1016/j.ins.2022.09.062_b0075) 2014 |
| References_xml | – reference: T. Li, A.K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith. Federated optimization in heterogeneous networks. arXiv preprint arXiv:1812.06127, 2018. – reference: P. Zhou, X. Yuan, and J. Feng. Efficient stochastic gradient hard thresholding. In Advances in Neural Information Processing Systems, pages 1988–1997, 2018. – volume: 172 start-page: 1122 year: 2018 end-page: 1131 ident: b0110 article-title: Identifying medical diagnoses and treatable diseases by image-based deep learning publication-title: Cell – reference: R.B. Harikandeh, M.O. Ahmed, A. Virani, M. Schmidt, J. Konečný, and S. Sallinen. Stopwasting my gradients: Practical svrg. In Advances in Neural Information Processing Systems, pages 2251–2259, 2015. – volume: 503 start-page: 219 year: 2019 end-page: 237 ident: b0135 article-title: The optimal upper bound of the number of queries for laplace mechanism under differential privacy publication-title: Information Sciences – reference: A. Rényi et al. On measures of entropy and information. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics. The Regents of the University of California, 1961. – start-page: 308 year: 2016 end-page: 318 ident: b0010 article-title: Deep learning with differential privacy publication-title: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security – reference: L. Wang and Q. Gu. A knowledge transfer framework for differentially private sparse learning. arXiv preprint arXiv:1909.06322, 2019b. – volume: 602 start-page: 57 year: 2022 end-page: 74 ident: b0095 article-title: Pricing gan-based data generators under rényi differential privacy publication-title: Information Sciences – reference: L. Wang and Q. Gu. Differentially private iterative gradient hard thresholding for sparse learning. In 28th International Joint Conference on Artificial Intelligence, 2019a. – volume: 29 start-page: 141 year: 2012 end-page: 142 ident: b0045 article-title: The mnist database of handwritten digit images for machine learning research [best of the web] publication-title: IEEE Signal Processing Magazine – reference: J. Near. Differential privacy at scale: Uber and berkeley collaboration. In Enigma 2018 (Enigma 2018), 2018. – volume: 40 start-page: 793 year: 2011 end-page: 826 ident: b0105 article-title: What can we learn privately? publication-title: SIAM Journal on Computing – reference: L. Wang, B. Jayaraman, D. Evans, and Q. Gu. Efficient privacy-preserving nonconvex optimization. arXiv preprint arXiv:1910.13659, 2019. – volume: 12 start-page: 1 year: 2022 end-page: 10 ident: b0015 article-title: Federated learning and differential privacy for medical image analysis publication-title: Scientific reports – reference: X. Li, R. Arora, H. Liu, J. Haupt, and T. Zhao. Nonconvex sparse learning via stochastic optimization with progressive variance reduction. arXiv preprint arXiv:1605.02711, 2016a. – volume: 14 start-page: 807 year: 2013 end-page: 841 ident: b0020 article-title: Greedy sparsity-constrained optimization publication-title: Journal of Machine Learning Research – year: 2022 ident: b0080 article-title: Combinatorial resources auction in decentralized edge-thing systems using blockchain and differential privacy publication-title: Information Sciences – start-page: 486 year: 2006 end-page: 503 ident: b0050 article-title: Our data, ourselves: Privacy via distributed noise generation publication-title: Annual International Conference on the Theory and Applications of Cryptographic Techniques – start-page: 6628 year: 2019 end-page: 6637 ident: b0195 article-title: On sparse linear regression in the local differential privacy model publication-title: International Conference on Machine Learning – reference: D. Wang, M. Ye, and J. Xu. Differentially private empirical risk minimization revisited: Faster and more general. In Advances in Neural Information Processing Systems, pages 2722–2731, 2017. – reference: P. Jain, A. Tewari, and P. Kar. On iterative hard thresholding methods for high-dimensional m-estimation. In Advances in Neural Information Processing Systems, pages 685–693, 2014. – volume: 10 start-page: 22359 year: 2022 end-page: 22380 ident: b0065 article-title: Differential privacy for deep and federated learning: A survey publication-title: IEEE Access – reference: X. Li, T. Zhao, R. Arora, H. Liu, and J. Haupt. Stochastic variance reduced optimization for nonconvex sparse learning. In International Conference on Machine Learning, pages 917–925, 2016b. – reference: Q. Tong, G. Liang, T. Zhu, and J. Bi. Federated nonconvex sparse learning. arXiv preprint arXiv:2101.00052, 2020. – reference: D. Kifer, A. Smith, and A. Thakurta. Private convex empirical risk minimization and high-dimensional regression. In Conference on Learning Theory, pages 25–1, 2012. – volume: 12 start-page: 1069 year: 2011 end-page: 1109 ident: b0035 article-title: Differentially private empirical risk minimization publication-title: Journal of Machine Learning Research – reference: G. Liang, Q. Tong, C.J. Zhu, and J. Bi. An effective hard thresholding method based on stochastic variance reduction for nonconvex sparse learning. In AAAI, pages 1585–1592, 2020. – volume: 560 start-page: 347 year: 2021 end-page: 369 ident: b0205 article-title: Correlated tuple data release via differential privacy publication-title: Information Sciences – start-page: 1307 year: 2017 end-page: 1322 ident: b0230 article-title: Bolt-on differential privacy for scalable stochastic gradient descent-based analytics publication-title: Proceedings of the 2017 ACM International Conference on Management of Data – reference: M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G.S. Corrado, A. Davis, J. Dean, M. Devin, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467, 2016a. – reference: A.E.C. Cloud. Amazon web services. Retrieved November, 9 (2011): 2011, 2011. – volume: 63 start-page: 6869 year: 2017 end-page: 6895 ident: b0165 article-title: Linear convergence of stochastic iterative greedy algorithms with sparse constraints publication-title: IEEE Transactions on Information Theory – start-page: 265 year: 2006 end-page: 284 ident: b0055 article-title: Calibrating noise to sensitivity in private data analysis publication-title: Theory of cryptography conference – start-page: 1054 year: 2014 end-page: 1067 ident: b0075 article-title: Rappor: Randomized aggregatable privacy-preserving ordinal response publication-title: Proceedings of the 2014 ACM SIGSAC conference on computer and communications security – reference: L. Lei and M. Jordan. Less than a single pass: Stochastically controlled stochastic gradient. In Artificial Intelligence and Statistics, pages 148–156, 2017. – volume: 27 start-page: 265 year: 2009 end-page: 274 ident: b0030 article-title: Iterative hard thresholding for compressed sensing publication-title: Applied and computational harmonic analysis – start-page: 51 year: 2010 end-page: 60 ident: b0060 article-title: Boosting and differential privacy publication-title: 2010 IEEE 51st Annual Symposium on Foundations of Computer Science – reference: M. Elibol, L. Lei, and M.I. Jordan. Variance reduction with sparse gradients. arXiv preprint arXiv:2001.09623, 2020. – volume: 41 start-page: 3397 year: 1993 end-page: 3415 ident: b0150 article-title: Matching pursuits with time-frequency dictionaries publication-title: IEEE Transactions on signal processing – reference: Y.-X. Wang, B. Balle, and S. Kasiviswanathan. Subsampled r⧹)ényi differential privacy and analytical moments accountant. arXiv preprint arXiv:1808.00087, 2018. – reference: K. Talwar, A.G. Thakurta, and L. Zhang. Nearly optimal private lasso. In Advances in Neural Information Processing Systems, pages 3025–3033, 2015. – reference: R. Johnson and T. Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In Advances in neural information processing systems, pages 315–323, 2013. – start-page: 464 year: 2014 end-page: 473 ident: b0025 article-title: Private empirical risk minimization: Efficient algorithms and tight error bounds publication-title: 2014 IEEE 55th Annual Symposium on Foundations of Computer Science – year: 2019 ident: b0190 article-title: Demystifying membership inference attacks in machine learning as a service publication-title: IEEE Transactions on Services Computing – start-page: 263 year: 2017 end-page: 275 ident: b0155 article-title: Rényi differential privacy publication-title: 2017 IEEE 30th Computer Security Foundations Symposium (CSF) – start-page: 40 year: 1993 end-page: 44 ident: b0170 article-title: Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition publication-title: Proceedings of 27th Asilomar conference on signals, systems and computers – start-page: 621 year: 2009 end-page: 625 ident: b0240 article-title: Ibm cloud computing powering a smarter planet publication-title: IEEE International Conference on Cloud Computing – volume: 29 start-page: 141 issue: 6 year: 2012 ident: 10.1016/j.ins.2022.09.062_b0045 article-title: The mnist database of handwritten digit images for machine learning research [best of the web] publication-title: IEEE Signal Processing Magazine doi: 10.1109/MSP.2012.2211477 – ident: 10.1016/j.ins.2022.09.062_b0180 – ident: 10.1016/j.ins.2022.09.062_b0235 – ident: 10.1016/j.ins.2022.09.062_b0130 – ident: 10.1016/j.ins.2022.09.062_b0210 doi: 10.24963/ijcai.2019/519 – volume: 40 start-page: 793 issue: 3 year: 2011 ident: 10.1016/j.ins.2022.09.062_b0105 article-title: What can we learn privately? publication-title: SIAM Journal on Computing doi: 10.1137/090756090 – ident: 10.1016/j.ins.2022.09.062_b0040 – start-page: 486 year: 2006 ident: 10.1016/j.ins.2022.09.062_b0050 article-title: Our data, ourselves: Privacy via distributed noise generation – volume: 172 start-page: 1122 issue: 5 year: 2018 ident: 10.1016/j.ins.2022.09.062_b0110 article-title: Identifying medical diagnoses and treatable diseases by image-based deep learning publication-title: Cell doi: 10.1016/j.cell.2018.02.010 – ident: 10.1016/j.ins.2022.09.062_b0120 – ident: 10.1016/j.ins.2022.09.062_b0225 – start-page: 621 year: 2009 ident: 10.1016/j.ins.2022.09.062_b0240 article-title: Ibm cloud computing powering a smarter planet – ident: 10.1016/j.ins.2022.09.062_b0070 – ident: 10.1016/j.ins.2022.09.062_b0200 – ident: 10.1016/j.ins.2022.09.062_b0185 – volume: 12 start-page: 1 issue: 1 year: 2022 ident: 10.1016/j.ins.2022.09.062_b0015 article-title: Federated learning and differential privacy for medical image analysis publication-title: Scientific reports doi: 10.1038/s41598-022-05539-7 – volume: 27 start-page: 265 issue: 3 year: 2009 ident: 10.1016/j.ins.2022.09.062_b0030 article-title: Iterative hard thresholding for compressed sensing publication-title: Applied and computational harmonic analysis doi: 10.1016/j.acha.2009.04.002 – volume: 10 start-page: 22359 year: 2022 ident: 10.1016/j.ins.2022.09.062_b0065 article-title: Differential privacy for deep and federated learning: A survey publication-title: IEEE Access doi: 10.1109/ACCESS.2022.3151670 – start-page: 1054 year: 2014 ident: 10.1016/j.ins.2022.09.062_b0075 article-title: Rappor: Randomized aggregatable privacy-preserving ordinal response – year: 2022 ident: 10.1016/j.ins.2022.09.062_b0080 article-title: Combinatorial resources auction in decentralized edge-thing systems using blockchain and differential privacy publication-title: Information Sciences doi: 10.1016/j.ins.2022.05.128 – start-page: 40 year: 1993 ident: 10.1016/j.ins.2022.09.062_b0170 article-title: Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition – start-page: 308 year: 2016 ident: 10.1016/j.ins.2022.09.062_b0010 article-title: Deep learning with differential privacy – ident: 10.1016/j.ins.2022.09.062_b0215 – start-page: 464 year: 2014 ident: 10.1016/j.ins.2022.09.062_b0025 article-title: Private empirical risk minimization: Efficient algorithms and tight error bounds – volume: 12 start-page: 1069 issue: Mar year: 2011 ident: 10.1016/j.ins.2022.09.062_b0035 article-title: Differentially private empirical risk minimization publication-title: Journal of Machine Learning Research – ident: 10.1016/j.ins.2022.09.062_b0160 – ident: 10.1016/j.ins.2022.09.062_b0005 – start-page: 51 year: 2010 ident: 10.1016/j.ins.2022.09.062_b0060 article-title: Boosting and differential privacy – volume: 560 start-page: 347 year: 2021 ident: 10.1016/j.ins.2022.09.062_b0205 article-title: Correlated tuple data release via differential privacy publication-title: Information Sciences doi: 10.1016/j.ins.2021.01.058 – ident: 10.1016/j.ins.2022.09.062_b0175 – start-page: 265 year: 2006 ident: 10.1016/j.ins.2022.09.062_b0055 article-title: Calibrating noise to sensitivity in private data analysis – ident: 10.1016/j.ins.2022.09.062_b0100 – start-page: 6628 year: 2019 ident: 10.1016/j.ins.2022.09.062_b0195 article-title: On sparse linear regression in the local differential privacy model – volume: 602 start-page: 57 year: 2022 ident: 10.1016/j.ins.2022.09.062_b0095 article-title: Pricing gan-based data generators under rényi differential privacy publication-title: Information Sciences doi: 10.1016/j.ins.2022.04.030 – volume: 503 start-page: 219 year: 2019 ident: 10.1016/j.ins.2022.09.062_b0135 article-title: The optimal upper bound of the number of queries for laplace mechanism under differential privacy publication-title: Information Sciences doi: 10.1016/j.ins.2019.07.001 – start-page: 1307 year: 2017 ident: 10.1016/j.ins.2022.09.062_b0230 article-title: Bolt-on differential privacy for scalable stochastic gradient descent-based analytics – year: 2019 ident: 10.1016/j.ins.2022.09.062_b0190 article-title: Demystifying membership inference attacks in machine learning as a service publication-title: IEEE Transactions on Services Computing – volume: 41 start-page: 3397 issue: 12 year: 1993 ident: 10.1016/j.ins.2022.09.062_b0150 article-title: Matching pursuits with time-frequency dictionaries publication-title: IEEE Transactions on signal processing doi: 10.1109/78.258082 – ident: 10.1016/j.ins.2022.09.062_b0085 – volume: 14 start-page: 807 issue: Mar year: 2013 ident: 10.1016/j.ins.2022.09.062_b0020 article-title: Greedy sparsity-constrained optimization publication-title: Journal of Machine Learning Research – start-page: 263 year: 2017 ident: 10.1016/j.ins.2022.09.062_b0155 article-title: Rényi differential privacy – ident: 10.1016/j.ins.2022.09.062_b0115 – ident: 10.1016/j.ins.2022.09.062_b0140 – ident: 10.1016/j.ins.2022.09.062_b0125 – ident: 10.1016/j.ins.2022.09.062_b0145 doi: 10.1609/aaai.v34i02.5519 – ident: 10.1016/j.ins.2022.09.062_b0220 – ident: 10.1016/j.ins.2022.09.062_b0090 – volume: 63 start-page: 6869 issue: 11 year: 2017 ident: 10.1016/j.ins.2022.09.062_b0165 article-title: Linear convergence of stochastic iterative greedy algorithms with sparse constraints publication-title: IEEE Transactions on Information Theory doi: 10.1109/TIT.2017.2749330 |
| SSID | ssj0004766 |
| Score | 2.4110239 |
| Snippet | Sparse learning is essential in mining high-dimensional data. Iterative hard thresholding (IHT) methods are effective for optimizing nonconvex objectives for... |
| SourceID | crossref elsevier |
| SourceType | Index Database Publisher |
| StartPage | 567 |
| SubjectTerms | Differential privacy Sparse learning Stochastic algorithm |
| Title | Stochastic privacy-preserving methods for nonconvex sparse learning |
| URI | https://dx.doi.org/10.1016/j.ins.2022.09.062 |
| Volume | 630 |
| WOSCitedRecordID | wos000946723000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1872-6291 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0004766 issn: 0020-0255 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1La9tAEF6C3UN7KHmVpnmwh9JDzIKsxz6OJg-SUEJKHfBNrDa7iXNQHNsJzr_PSDt6NKmhPvQizBqNbM3HzDe78yDke5ZFlnMZsUAoxWJpFdMycyyIXN8BIQev5cphE-LyUo5G6grTbWflOAGR53KxUJP_qmpYA2UXpbMrqLsWCgvwGZQOV1A7XP9J8b_nD-ZOF-2Xe5Pp-FmbF1YkuxY2Ib_FidFlE4YeRP5l0vmiB2ZlOrPVCInbNmPFeqUSJuguaxr-c4y7zQC0PG9wNsQ831_w_WMLf8c4QOVirO_0Q3N6hQn8uIR7EGHU5ErVNQEBK6KTtl3leODiLWPip26gk038nJ539ttvJdxD0FG0Ug_Dsgctmus_emW_8WF1ZmGVtHafgoi0EJEGKg0KL90NRaJkh3QH5yeji6Z4VvgD7eovVEffZRLgm9_xd_LSIiTDdfIZIwk68AjYIGs23ySfWv0lN8k-VqXQH7SlRor2fIscNVih77FCESsU7qQ1VqjHCq2wsk2uT0-GR2cMp2owA6H2nMVOh8ImVt24RDgnuNaGOwtkCbiyzETArXORM4HgEDpkRjkVm8wG_Sy0Ukb96AvpwDPtV0KBLWugs8ZEkscC_BXXIFTeAMcH8SHfIYfV60onvnlKulRBOySuXmiKcPasLgVwLL_t2yrP2CUfG_jukc58-mT3yQfzPB_PpgeIjFfCX3zG |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Stochastic+privacy-preserving+methods+for+nonconvex+sparse+learning&rft.jtitle=Information+sciences&rft.au=Liang%2C+Guannan&rft.au=Tong%2C+Qianqian&rft.au=Ding%2C+Jiahao&rft.au=Pan%2C+Miao&rft.date=2023-06-01&rft.issn=0020-0255&rft.volume=630&rft.spage=567&rft.epage=585&rft_id=info:doi/10.1016%2Fj.ins.2022.09.062&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_ins_2022_09_062 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0020-0255&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0020-0255&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0020-0255&client=summon |