Zeroth-Order Nonconvex Stochastic Optimization: Handling Constraints, High Dimensionality, and Saddle Points

In this paper, we propose and analyze zeroth-order stochastic approximation algorithms for nonconvex and convex optimization, with a focus on addressing constrained optimization, high-dimensional setting, and saddle point avoiding. To handle constrained optimization, we first propose generalizations...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	Foundations of computational mathematics Ročník 22; číslo 1; s. 35 - 76
Hlavní autori:	Balasubramanian, Krishnakumar, Ghadimi, Saeed
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	New York Springer US 01.02.2022 Springer Nature B.V
Predmet:	Algorithms Applications of Mathematics Computational geometry Computer Science Constraints Convergence Convexity Economics Hessian matrices Image processing Linear and Multilinear Algebras Math Applications in Computer Science Mathematical analysis Mathematics Mathematics and Statistics Matrix Theory Newton methods Numerical Analysis Optimization Regularization Saddle points Smoothing Sparsity Nonconvex optimization 65K05 90C56 Conditional gradient methods Complexity bounds 90C15 90C26 49M15 Stochastic optimization Zeroth-order methods Newton method
ISSN:	1615-3375, 1615-3383
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	In this paper, we propose and analyze zeroth-order stochastic approximation algorithms for nonconvex and convex optimization, with a focus on addressing constrained optimization, high-dimensional setting, and saddle point avoiding. To handle constrained optimization, we first propose generalizations of the conditional gradient algorithm achieving rates similar to the standard stochastic gradient algorithm using only zeroth-order information. To facilitate zeroth-order optimization in high dimensions, we explore the advantages of structural sparsity assumptions. Specifically, (i) we highlight an implicit regularization phenomenon where the standard stochastic gradient algorithm with zeroth-order information adapts to the sparsity of the problem at hand by just varying the step size and (ii) propose a truncated stochastic gradient algorithm with zeroth-order information, whose rate of convergence depends only poly-logarithmically on the dimensionality. We next focus on avoiding saddle points in nonconvex setting. Toward that, we interpret the Gaussian smoothing technique for estimating gradient based on zeroth-order information as an instantiation of first-order Stein’s identity. Based on this, we provide a novel linear-(in dimension) time estimator of the Hessian matrix of a function using only zeroth-order information, which is based on second-order Stein’s identity. We then provide a zeroth-order variant of cubic regularized Newton method for avoiding saddle points and discuss its rate of convergence to local minima.
AbstractList	In this paper, we propose and analyze zeroth-order stochastic approximation algorithms for nonconvex and convex optimization, with a focus on addressing constrained optimization, high-dimensional setting, and saddle point avoiding. To handle constrained optimization, we first propose generalizations of the conditional gradient algorithm achieving rates similar to the standard stochastic gradient algorithm using only zeroth-order information. To facilitate zeroth-order optimization in high dimensions, we explore the advantages of structural sparsity assumptions. Specifically, (i) we highlight an implicit regularization phenomenon where the standard stochastic gradient algorithm with zeroth-order information adapts to the sparsity of the problem at hand by just varying the step size and (ii) propose a truncated stochastic gradient algorithm with zeroth-order information, whose rate of convergence depends only poly-logarithmically on the dimensionality. We next focus on avoiding saddle points in nonconvex setting. Toward that, we interpret the Gaussian smoothing technique for estimating gradient based on zeroth-order information as an instantiation of first-order Stein’s identity. Based on this, we provide a novel linear-(in dimension) time estimator of the Hessian matrix of a function using only zeroth-order information, which is based on second-order Stein’s identity. We then provide a zeroth-order variant of cubic regularized Newton method for avoiding saddle points and discuss its rate of convergence to local minima.
Author	Ghadimi, Saeed Balasubramanian, Krishnakumar
Author_xml	– sequence: 1 givenname: Krishnakumar surname: Balasubramanian fullname: Balasubramanian, Krishnakumar organization: Department of Statistics, University of California – sequence: 2 givenname: Saeed surname: Ghadimi fullname: Ghadimi, Saeed email: sghadimi@uwaterloo.ca organization: Department of Management Sciences, University of Waterloo
BookMark	eNp9kE1LAzEQhoNU0Fb_gKeAV1fzsdlkvUn9qFCsUL14CWmSbSPbpCZR1F_v1oqCh8LAzOF5hpm3D3o-eAvAEUanGCF-ljAiSBSI4ALVZV0XYgfs4wqzglJBe78zZ3ugn9IzQpjVuNwH7ZONIS-KSTQ2wrvgdfBv9h1Oc9ALlbLTcLLKbuk-VXbBn8OR8qZ1fg6HwacclfM5ncCRmy_gpVtanzpKtS5_nMCOhFNlTGvhfVhzB2C3UW2yhz99AB6vrx6Go2I8ubkdXowLXZIyF1xjMRNUqFKUnNSVtqw2hHTFMaPUUG4IE1XTCGOt0TPESz6boUowzhHmDR2A483eVQwvrzZl-RxeY3dWkqQijHHBWdlRYkPpGFKKtpHa5e8v12-1EiO5zlZuspVdtvI7Wyk6lfxTV9EtVfzYLtGNlDrYz238u2qL9QU9So6L
CitedBy_id	crossref_primary_10_1007_s10915_025_02855_8 crossref_primary_10_1007_s10957_025_02647_y crossref_primary_10_1287_ijoc_2023_0247 crossref_primary_10_1287_moor_2022_1302 crossref_primary_10_1016_j_automatica_2024_111528 crossref_primary_10_1016_j_neucom_2024_128606 crossref_primary_10_1287_ijoo_2022_0085 crossref_primary_10_1007_s10957_021_01994_w crossref_primary_10_1007_s12532_023_00233_9 crossref_primary_10_1007_s10898_022_01160_0 crossref_primary_10_1146_annurev_control_042920_020021 crossref_primary_10_1109_TAC_2023_3341752 crossref_primary_10_1109_LCSYS_2023_3282647 crossref_primary_10_1137_22M1510261 crossref_primary_10_1007_s40305_025_00584_1 crossref_primary_10_1007_s10915_025_03066_x crossref_primary_10_1016_j_eswa_2024_125781 crossref_primary_10_1109_JIOT_2025_3538508 crossref_primary_10_1137_24M1659911 crossref_primary_10_3150_21_BEJ1400 crossref_primary_10_1137_22M1494270 crossref_primary_10_1007_s10287_023_00479_7
Cites_doi	10.1016/j.jco.2019.101443 10.1017/CBO9780511804441 10.1007/s10959-008-0155-9 10.1016/j.jco.2011.01.004 10.1007/s10208-017-9365-9 10.1137/140992382 10.1214/17-AOS1642 10.1002/9781118631980 10.1134/S0005117917020035 10.1093/imaiai/iax008 10.1137/1.9781611974997 10.1007/s10107-009-0286-5 10.1007/s10208-015-9296-2 10.1007/s10107-009-0337-y 10.1007/s10107-019-01405-z 10.1109/TIT.2006.871582 10.1007/978-1-4419-8853-9 10.1007/978-3-319-40519-3_8 10.1137/17M1114296 10.1561/2200000024 10.1145/3128572.3140448 10.1109/ALLERTON.2016.7852377 10.1137/120880811 10.1007/s10107-017-1225-5 10.1007/s10107-019-01449-1 10.1137/1.9780898718768 10.1007/s00365-010-9105-8 10.1002/nav.3800030109 10.1561/2200000058 10.1145/3055399.3055403 10.1007/s10208-017-9363-y 10.1137/1.9780898718829 10.1214/aos/1176345632 10.1007/BF02592948 10.1007/s10107-006-0706-8 10.1109/TIT.2015.2409256
ContentType	Journal Article
Copyright	SFoCM 2021 SFoCM 2021.
Copyright_xml	– notice: SFoCM 2021 – notice: SFoCM 2021.
DBID	AAYXX CITATION 7SC 7TB 8FD FR3 JQ2 KR7 L7M L~C L~D
DOI	10.1007/s10208-021-09499-8
DatabaseName	CrossRef Computer and Information Systems Abstracts Mechanical & Transportation Engineering Abstracts Technology Research Database Engineering Research Database ProQuest Computer Science Collection Civil Engineering Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Civil Engineering Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Mechanical & Transportation Engineering Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Civil Engineering Abstracts
DeliveryMethod	fulltext_linktorsrc
Discipline	Economics Mathematics Applied Sciences Computer Science
EISSN	1615-3383
EndPage	76
ExternalDocumentID	10_1007_s10208_021_09499_8
GroupedDBID	-5D -5G -BR -EM -Y2 -~C .4S .86 .DC .VR 06D 0R~ 0VY 1N0 1SB 203 29H 2J2 2JN 2JY 2KG 2LR 2P1 2VQ 2~H 30V 4.4 406 408 409 40D 40E 5GY 5VS 67Z 6NX 8TC 8UJ 95- 95. 95~ 96X AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDBF ABDZT ABECU ABFTD ABFTV ABHLI ABHQN ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTEG ABTHY ABTKH ABTMW ABULA ABWNU ABXPI ACAOD ACBXY ACDTI ACGFS ACGOD ACHSB ACHXU ACIWK ACKNC ACMDZ ACMLO ACOKC ACOMO ACPIV ACSNA ACUHS ACZOJ ADHHG ADHIR ADINQ ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF ADZKW AEBTG AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFGCZ AFLOW AFQWF AFWTZ AFZKB AGAYW AGDGC AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMXSW AMYLF AMYQR AOCGG ARCSS ARMRJ ASPBG AVWKF AXYYD AYJHY AZFZN B-. B0M BA0 BAPOH BDATZ BGNMA BSONS CAG COF CS3 CSCUP DDRTE DL5 DNIVK DPUIP DU5 EAD EAP EBLON EBS EDO EIOEI EJD EMK EPL ESBYG ESX FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNWQR GQ6 GQ7 GQ8 GXS H13 HF~ HG5 HG6 HLICF HMJXF HQYDN HRMNR HVGLF HZ~ I-F IAO IEA IHE IJ- IKXTQ IOF ISR ITC ITM IWAJR IXC IZIGR IZQ I~X I~Z J-C J0Z J9A JBSCW JCJTX JZLTJ KDC KOV LAS LLZTM M4Y MA- MK~ N2Q N9A NPVJJ NQJWS NU0 O9- O93 O9J OAM P2P P9R PF0 PQQKQ PT4 Q2X QOS R89 R9I RIG ROL RPX RSV S16 S1Z S27 S3B SAP SDH SHX SISQX SJYHP SMT SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 TSG TSK TSV TUC TUS U2A UG4 UOJIU UTJUX UZXMN VC2 VFIZW W23 W48 WK8 YLTOR Z45 Z81 Z83 Z88 ZMTXR ~8M AAPKM AAYXX ABBRH ABDBE ABFSG ABRTQ ACSTC ADHKG AEZWR AFDZB AFHIU AFOHR AGQPQ AHPBZ AHWEU AIXLP AMVHM ATHPR AYFIA CITATION ICD 7SC 7TB 8FD FR3 JQ2 KR7 L7M L~C L~D
ID	FETCH-LOGICAL-c424t-7c18b838a4847296ce59d22d2271533d37d2586ff8deedcb0747bb068577017f3
IEDL.DBID	RSV
ISICitedReferencesCount	44
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000630642400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1615-3375
IngestDate	Fri Jul 25 19:30:56 EDT 2025 Sat Nov 29 04:08:43 EST 2025 Tue Nov 18 22:43:18 EST 2025 Fri Feb 21 02:47:55 EST 2025
IsPeerReviewed	true
IsScholarly	true
Issue	1
Keywords	Nonconvex optimization 65K05 90C56 Conditional gradient methods Complexity bounds 90C15 90C26 49M15 Stochastic optimization Zeroth-order methods Newton method
Language	English
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c424t-7c18b838a4847296ce59d22d2271533d37d2586ff8deedcb0747bb068577017f3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
PQID	2625578754
PQPubID	43692
PageCount	42
ParticipantIDs	proquest_journals_2625578754 crossref_citationtrail_10_1007_s10208_021_09499_8 crossref_primary_10_1007_s10208_021_09499_8 springer_journals_10_1007_s10208_021_09499_8
PublicationCentury	2000
PublicationDate	2022-02-01
PublicationDateYYYYMMDD	2022-02-01
PublicationDate_xml	– month: 02 year: 2022 text: 2022-02-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationSubtitle	The Journal of the Society for the Foundations of Computational Mathematics
PublicationTitle	Foundations of computational mathematics
PublicationTitleAbbrev	Found Comput Math
PublicationYear	2022
Publisher	Springer US Springer Nature B.V
Publisher_xml	– name: Springer US – name: Springer Nature B.V
References	XuPRoosta-KhorasaniFMahoneyMWNewton-type methods for non-convex optimization under inexact hessian informationMathematical Programming20201843570416354010.1007/s10107-019-01405-z Mockus, J.: Bayesian approach to global optimization: theory and applications, vol. 37. Springer Science & Business Media (2012) RioEMoment inequalities for sums of dependent random variables under projective conditionsJournal of Theoretical Probability2009221146163247201010.1007/s10959-008-0155-9 Conn, A., Scheinberg, K., Vicente, L.: Introduction to derivative-free optimization, vol. 8. Society of Industrial and Applied Mathematics (SIAM) (2009) BoydSVandenbergheLConvex optimization2004CambridgeCambridge University Press10.1017/CBO9780511804441 Akhavan, A., Pontil, M., Tsybakov, A.: Exploiting higher order smoothness in derivative-free optimization and continuous bandits. In: Advances in Neural Information Processing Systems, vol. 33 (2020) CartisCGouldNITointPLSecond-order optimality and beyond: Characterization and evaluation complexity in convexly constrained nonlinear optimizationFoundations of Computational Mathematics201818510731107385790510.1007/s10208-017-9363-y FrankMWolfePAn algorithm for quadratic programmingNaval Research Logistics Quarterly19563951108910210.1002/nav.3800030109 Nocedal, J., Wright, S.J.: Numerical optimization. Springer Science & Business Media (2006) Bubeck, S., Cesa-Bianchi, N.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Hu, X., Prashanth, L.A., György, A., Szepesvari, C.: (Bandit) Convex Optimization with Biased Noisy Gradient Oracles. In: The 19th International Conference on Artificial Intelligence and Statistics, pp. 3420–3428 (2016) Mokhtari, A., Hassani, H., Karbasi, A.: Conditional gradient method for stochastic submodular maximization: Closing the gap. In: International Conference on Artificial Intelligence and Statistics, pp. 1886–1895 (2018) Ge, R., Lee, J.D., Ma, T.: Matrix completion has no spurious local minimum. In: Advances in Neural Information Processing Systems, pp. 2973–2981 (2016) DeVoreRPetrovaGWojtaszczykPApproximation of functions of few variables in high dimensionsConstructive Approximation2011331125143274705910.1007/s00365-010-9105-8 Nesterov, Y.: Introductory lectures on convex optimization: A basic course, vol. 87. Springer Science & Business Media (2013) GhadimiSLanGStochastic first- and zeroth-order methods for nonconvex stochastic programmingSIAM Journal on Optimization201323423412368313443910.1137/120880811 Bertsekas, D.P., Scientific, A.: Convex optimization algorithms. Athena Scientific Belmont (2015) Dani, V., Kakade, S.M., Hayes, T.P.: The price of bandit information for online optimization. In: Advances in Neural Information Processing Systems, pp. 345–352 (2008) ErdogduMANewton-Stein method: an optimization method for GLMs via Stein’s lemmaThe Journal of Machine Learning Research20161717565761635951501437.62292 MokhtariAHassaniHKarbasiAStochastic conditional gradient methods: From convex minimization to submodular maximizationJournal of Machine Learning Research202021149411917307255136 Reddi, S., Sra, S., Póczos, B., Smola, A.: Stochastic Frank-Wolfe Methods for Nonconvex Optimization. In: Proceedings of the 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1244–1251 (2016) Sun, J., Qu, Q., Wright, J.: When are nonconvex problems not scary? arXiv:1510.06096 Snoek, J., Larochelle, H., Adams, R.: Practical bayesian optimization of machine learning algorithms. In: Advances in neural information processing systems, pp. 2951–2959 (2012) Demyanov, V., Rubinov, A.: Approximate methods in optimization problems. American Elsevier Publishing (1970) Elibol, M., Lei, L., Jordan, M.I.: Variance reduction with sparse gradients. In: Proceedings of the 8th International Conference on Learning Representations (ICLR), pp. 1058–1070 (2020) HanCYuanMInformation based complexity for high dimensional sparse functionsJournal of Complexity202057101443405505810.1016/j.jco.2019.101443 Hazan, E., Kale, S.: Projection-free online learning. In: Proceedings of the 29th International Conference on International Conference on Machine Learning, pp. 1843–1850 (2012) Jin, C., Ge, R., Netrapalli, P., Kakade, S.M., Jordan, M.I.: How to escape saddle points efficiently. In: International Conference on Machine Learning, pp. 1724–1732 (2017) CartisCGouldNITointPLAdaptive cubic regularisation methods for unconstrained optimization, Part I: Motivation, convergence and numerical resultsMathematical Programming20111272245295277670110.1007/s10107-009-0286-5 NesterovYPolyakBCubic regularization of newton method and its global performanceMathematical Programming20061081177205222945910.1007/s10107-006-0706-8 Bertsekas, D.P.: Nonlinear programming. Athena scientific Belmont (2016) Stein, C.: A bound for the error in the normal approximation to the distribution of a sum of dependent random variables. In: Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Volume 2: Probability Theory. The Regents of the University of California (1972) Cai, H., Mckenzie, D., Yin, W., Zhang, Z.: Zeroth-order regularized optimization (ZORO): Approximately sparse gradients and adaptive sampling (2020) SpallJIntroduction to stochastic search and optimization: estimation, simulation, and control2005New JerseyJohn Wiley & Sons1088.90002 DuchiJJordanMWainwrightMWibisonoAOptimal rates for zero-order convex optimization: The power of two function evaluationsIEEE Transactions on Information Theory201561527882806334230710.1109/TIT.2015.2409256 Chen, L., Zhang, M., Hassani, H., Karbasi, A.: Black box submodular maximization: Discrete and continuous settings. In: S. Chiappa, R. Calandra (eds.) Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, vol. 108, pp. 1058–1070 (2020) RubinsteinRKroeseDSimulation and the Monte Carlo method2016New JerseyJohn Wiley & Sons10.1002/9781118631980 Bach, F., Perchet, V.: Highly-smooth zero-th order online optimization. In: V. Feldman, A. Rakhlin, O. Shamir (eds.) 29th Annual Conference on Learning Theory, Proceedings of Machine Learning Research, vol. 49, pp. 257–283. PMLR (2016) Reddi, S., Zaheer, M., Sra, S., Poczos, B., Bach, F., Salakhutdinov, R., Smola, A.: A generic approach for escaping saddle points. In: International Conference on Artificial Intelligence and Statistics, pp. 1233–1242 (2018) Jaggi, M.: Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization. In: Proceedings of the 30th International Conference on International Conference on Machine Learning, pp. 427–435 (2013) Tropp, J.A.: The expected norm of a sum of independent random matrices: An elementary approach. In: High Dimensional Probability VII, pp. 173–202. Springer (2016) Hazan, E., Levy, K.: Bandit convex optimization: Towards tight bounds. In: Advances in Neural Information Processing Systems, pp. 784–792 (2014) Beck, A.: First-Order Methods in Optimization, vol. 25. Society for Industrial and Applied Mathematics (SIAM) (2017) Agarwal, A., Dekel, O., Xiao, L.: Optimal algorithms for online convex optimization with multi-point bandit feedback. In: Proceedings of The 23rd Conference on Learning Theory, pp. 28–40 (2010) CarmonYDuchiJCHinderOSidfordAAccelerated methods for nonconvex optimizationSIAM Journal on Optimization201828217511772381402710.1137/17M1114296 WojtaszczykPComplexity of approximation of functions of few variables in high dimensionsJournal of Complexity2011272141150277648910.1016/j.jco.2011.01.004 Belloni, A., Liang, T., Narayanan, H., Rakhlin, A.: Escaping the local minima via simulated annealing: Optimization of approximately convex functions. In: P. Grunwald, E. Hazan, S. Kale (eds.) Proceedings of The 28th Conference on Learning Theory, Proceedings of Machine Learning Research, vol. 40, pp. 240–265. PMLR (2015) Jain, P., Kar, P.: Non-convex optimization for machine learning.Foundations and Trends® in Machine Learning 10(3-4), 142–336 (2017) Lattimore, T.: Improved regret for zeroth-order adversarial bandit convex optimisation. arXiv:2006.00475 Tripuraneni, N., Stern, M., Jin, C., Regier, J., Jordan, M.: Stochastic cubic regularization for fast nonconvex optimization. In: Advances in Neural Information Processing Systems, pp. 2899–2908 (2018) Shamir, O.: On the complexity of bandit and derivative-free stochastic convex optimization. In: Conference on Learning Theory, pp. 3–24 (2013) HearnDThe gap function of a convex programOperations Research Letters19822951100486.90070 Stein, C.M.: Estimation of the mean of a multivariate normal distribution. The annals of Statistics pp. 1135–1151 (1981) Jamieson, K., Nowak, R., Recht, B.: Query complexity of derivative-free optimization. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2012) NesterovYIntroductory Lectures on Convex Optimization: a basic course2004MassachusettsKluwer Academic Publishers10.1007/978-1-4419-8853-9 Chen, P.Y., Zhang, H., Sharma, Y., Yi, J., Hsieh, C.J.: ZOO: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 15–26. ACM (2017) Li, J., Balasubramanian, K., Ma, S.: Stochastic zeroth-order riemannian derivative estimation and optimization. arXiv:2003.11238 (2020) LanGZhouYConditional gradient sliding for convex optimizationSIAM Journal on Optimization201626213791409351687110.1137/140992382 Hazan, E., Luo, H.: Variance-reduced and projection-free stochastic optimization. In: International Conference on Machine Learning, pp. 1263–1271 (2016) TyagiHKyrillidisAGärtnerBKrauseAAlgorithms for learning sparse additive models with interactions in high dimensionsInformation and Inference: A Journal of the IMA201872183249381212010.1093/imaiai/iax008 SunJQuQWrightJA geometric analysis of phase retrieva Y Nestrov (9499_CR62) 2021; 186 9499_CR39 9499_CR38 AS Nemirovski (9499_CR57) 1983 Y Carmon (9499_CR15) 2018; 28 9499_CR33 D Hearn (9499_CR40) 1982; 2 9499_CR32 9499_CR76 9499_CR31 9499_CR75 9499_CR74 9499_CR37 C Cartis (9499_CR18) 2018; 18 C Han (9499_CR36) 2020; 57 9499_CR79 R DeVore (9499_CR25) 2011; 33 9499_CR34 9499_CR78 S Ghadimi (9499_CR35) 2013; 23 9499_CR72 J Spall (9499_CR73) 2005 9499_CR71 J Duchi (9499_CR27) 2015; 61 9499_CR70 9499_CR5 9499_CR6 9499_CR3 Y Nesterov (9499_CR58) 2004 9499_CR4 9499_CR9 9499_CR7 9499_CR8 9499_CR49 9499_CR1 9499_CR2 DL Donoho (9499_CR26) 2006; 52 9499_CR44 9499_CR43 9499_CR42 9499_CR41 R Rubinstein (9499_CR68) 2016 9499_CR47 P Xu (9499_CR83) 2020; 184 9499_CR46 9499_CR45 A Mokhtari (9499_CR55) 2020; 21 Y Nesterov (9499_CR61) 2017; 17 E Rio (9499_CR67) 2009; 22 9499_CR81 P Wojtaszczyk (9499_CR82) 2011; 27 S Minsker (9499_CR52) 2018; 46 9499_CR19 9499_CR10 9499_CR54 9499_CR53 9499_CR59 Y Nesterov (9499_CR60) 2006; 108 S Boyd (9499_CR11) 2004 9499_CR14 G Raskutti (9499_CR64) 2012; 13 9499_CR13 9499_CR12 G Lan (9499_CR48) 2016; 26 C Cartis (9499_CR16) 2011; 127 9499_CR51 9499_CR50 KG Murty (9499_CR56) 1987; 39 J Sun (9499_CR77) 2018; 18 H Tyagi (9499_CR80) 2018; 7 M Frank (9499_CR30) 1956; 3 9499_CR28 9499_CR22 9499_CR66 9499_CR21 9499_CR65 9499_CR20 9499_CR63 9499_CR69 9499_CR24 9499_CR23 C Cartis (9499_CR17) 2011; 130 MA Erdogdu (9499_CR29) 2016; 17
References_xml	– reference: CartisCGouldNITointPLAdaptive cubic regularisation methods for unconstrained optimization, Part II: Worst-case function-and derivative-evaluation complexityMathematical Programming20111302295319285587210.1007/s10107-009-0337-y – reference: BoydSVandenbergheLConvex optimization2004CambridgeCambridge University Press10.1017/CBO9780511804441 – reference: Hu, X., Prashanth, L.A., György, A., Szepesvari, C.: (Bandit) Convex Optimization with Biased Noisy Gradient Oracles. In: The 19th International Conference on Artificial Intelligence and Statistics, pp. 3420–3428 (2016) – reference: Jaggi, M.: Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization. In: Proceedings of the 30th International Conference on International Conference on Machine Learning, pp. 427–435 (2013) – reference: SunJQuQWrightJA geometric analysis of phase retrievalFoundations of Computational Mathematics201818511311198385790710.1007/s10208-017-9365-9 – reference: ErdogduMANewton-Stein method: an optimization method for GLMs via Stein’s lemmaThe Journal of Machine Learning Research20161717565761635951501437.62292 – reference: Ge, R., Huang, F., Jin, C., Yuan, Y.: Escaping from saddle points: Online stochastic gradient for tensor decomposition. In: Conference on Learning Theory, pp. 797–842 (2015) – reference: NesterovYIntroductory Lectures on Convex Optimization: a basic course2004MassachusettsKluwer Academic Publishers10.1007/978-1-4419-8853-9 – reference: Tropp, J.A.: The expected norm of a sum of independent random matrices: An elementary approach. In: High Dimensional Probability VII, pp. 173–202. Springer (2016) – reference: Ben-Tal, A., Nemirovski, A.: Lectures on modern convex optimization: analysis, algorithms, and engineering applications, vol. 2. Society for Industrial and Applied Mathematics (SIAM) (2001) – reference: Hazan, E., Kale, S.: Projection-free online learning. In: Proceedings of the 29th International Conference on International Conference on Machine Learning, pp. 1843–1850 (2012) – reference: Sun, J., Qu, Q., Wright, J.: When are nonconvex problems not scary? arXiv:1510.06096 – reference: Bubeck, S., Cesa-Bianchi, N.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) – reference: RubinsteinRKroeseDSimulation and the Monte Carlo method2016New JerseyJohn Wiley & Sons10.1002/9781118631980 – reference: Belloni, A., Liang, T., Narayanan, H., Rakhlin, A.: Escaping the local minima via simulated annealing: Optimization of approximately convex functions. In: P. Grunwald, E. Hazan, S. Kale (eds.) Proceedings of The 28th Conference on Learning Theory, Proceedings of Machine Learning Research, vol. 40, pp. 240–265. PMLR (2015) – reference: CarmonYDuchiJCHinderOSidfordAAccelerated methods for nonconvex optimizationSIAM Journal on Optimization201828217511772381402710.1137/17M1114296 – reference: Gasnikov, A.V., Krymova, E.A., Lagunovskaya, A.A., Usmanova, I.N., Fedorenko, F.A.: Stochastic online optimization. single-point and multi-point non-linear multi-armed bandits. convex and strongly-convex case. Automation and remote control 78(2), 224–234 (2017) – reference: Ge, R., Lee, J.D., Ma, T.: Matrix completion has no spurious local minimum. In: Advances in Neural Information Processing Systems, pp. 2973–2981 (2016) – reference: MinskerSSub-gaussian estimators of the mean of a random matrix with heavy-tailed entriesThe Annals of Statistics2018466A28712903385175810.1214/17-AOS1642 – reference: Hazan, E., Luo, H.: Variance-reduced and projection-free stochastic optimization. In: International Conference on Machine Learning, pp. 1263–1271 (2016) – reference: Bach, F., Perchet, V.: Highly-smooth zero-th order online optimization. In: V. Feldman, A. Rakhlin, O. Shamir (eds.) 29th Annual Conference on Learning Theory, Proceedings of Machine Learning Research, vol. 49, pp. 257–283. PMLR (2016) – reference: Bubeck, S., Lee, Y.T., Eldan, R.: Kernel-based methods for bandit convex optimization. In: Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, pp. 72–85 (2017) – reference: Allen-Zhu, Z.: Natasha 2: Faster non-convex optimization than SGD. In: Advances in Neural Information Processing Systems, pp. 2680–2691 (2018) – reference: CartisCGouldNITointPLSecond-order optimality and beyond: Characterization and evaluation complexity in convexly constrained nonlinear optimizationFoundations of Computational Mathematics201818510731107385790510.1007/s10208-017-9363-y – reference: Stein, C.M.: Estimation of the mean of a multivariate normal distribution. The annals of Statistics pp. 1135–1151 (1981) – reference: DuchiJJordanMWainwrightMWibisonoAOptimal rates for zero-order convex optimization: The power of two function evaluationsIEEE Transactions on Information Theory201561527882806334230710.1109/TIT.2015.2409256 – reference: Tripuraneni, N., Stern, M., Jin, C., Regier, J., Jordan, M.: Stochastic cubic regularization for fast nonconvex optimization. In: Advances in Neural Information Processing Systems, pp. 2899–2908 (2018) – reference: Bertsekas, D.P.: Nonlinear programming. Athena scientific Belmont (2016) – reference: Nesterov, Y.: Introductory lectures on convex optimization: A basic course, vol. 87. Springer Science & Business Media (2013) – reference: NestrovYImplementable tensor methods in unconstrained convex optimizationMathematical Programming2021186157183421447910.1007/s10107-019-01449-1 – reference: TyagiHKyrillidisAGärtnerBKrauseAAlgorithms for learning sparse additive models with interactions in high dimensionsInformation and Inference: A Journal of the IMA201872183249381212010.1093/imaiai/iax008 – reference: Nocedal, J., Wright, S.J.: Numerical optimization. Springer Science & Business Media (2006) – reference: RaskuttiGWainwrightMJYuBMinimax-optimal rates for sparse additive models over kernel classes via convex programmingThe Journal of Machine Learning Research201213138942729137041283.62071 – reference: Jamieson, K., Nowak, R., Recht, B.: Query complexity of derivative-free optimization. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2012) – reference: SpallJIntroduction to stochastic search and optimization: estimation, simulation, and control2005New JerseyJohn Wiley & Sons1088.90002 – reference: Agarwal, A., Dekel, O., Xiao, L.: Optimal algorithms for online convex optimization with multi-point bandit feedback. In: Proceedings of The 23rd Conference on Learning Theory, pp. 28–40 (2010) – reference: Saha, A., Tewari, A.: Improved regret guarantees for online smooth convex optimization with bandit feedback. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 636–642 (2011) – reference: MurtyKGKabadiSNSome NP-complete problems in quadratic and nonlinear programmingMathematical Programming198739211712991600110.1007/BF02592948 – reference: NemirovskiASYudinDProblem complexity and method efficiency in optimization1983XVWiley-Interscience Series in Discrete Mathematics. John Wiley – reference: XuPRoosta-KhorasaniFMahoneyMWNewton-type methods for non-convex optimization under inexact hessian informationMathematical Programming20201843570416354010.1007/s10107-019-01405-z – reference: Jain, P., Kar, P.: Non-convex optimization for machine learning.Foundations and Trends® in Machine Learning 10(3-4), 142–336 (2017) – reference: HanCYuanMInformation based complexity for high dimensional sparse functionsJournal of Complexity202057101443405505810.1016/j.jco.2019.101443 – reference: Demyanov, V., Rubinov, A.: Approximate methods in optimization problems. American Elsevier Publishing (1970) – reference: NesterovYSpokoinyVRandom gradient-free minimization of convex functionsFoundations of Computational Mathematics201717527566362745610.1007/s10208-015-9296-2 – reference: Dani, V., Kakade, S.M., Hayes, T.P.: The price of bandit information for online optimization. In: Advances in Neural Information Processing Systems, pp. 345–352 (2008) – reference: Jain, P., Tewari, A., Kar, P.: On iterative hard thresholding methods for high-dimensional m-estimation. In: Advances in Neural Information Processing Systems, pp. 685–693 (2014) – reference: Bertsekas, D.P., Scientific, A.: Convex optimization algorithms. Athena Scientific Belmont (2015) – reference: Ghadimi, S.: Conditional gradient type methods for composite nonlinear and stochastic optimization. Mathematical Programming (2018). https://doi.org/10.1007/s10107-017-1225-5 – reference: Lattimore, T.: Improved regret for zeroth-order adversarial bandit convex optimisation. arXiv:2006.00475 – reference: Conn, A., Scheinberg, K., Vicente, L.: Introduction to derivative-free optimization, vol. 8. Society of Industrial and Applied Mathematics (SIAM) (2009) – reference: Mokhtari, A., Hassani, H., Karbasi, A.: Conditional gradient method for stochastic submodular maximization: Closing the gap. In: International Conference on Artificial Intelligence and Statistics, pp. 1886–1895 (2018) – reference: Snoek, J., Larochelle, H., Adams, R.: Practical bayesian optimization of machine learning algorithms. In: Advances in neural information processing systems, pp. 2951–2959 (2012) – reference: WojtaszczykPComplexity of approximation of functions of few variables in high dimensionsJournal of Complexity2011272141150277648910.1016/j.jco.2011.01.004 – reference: Salimans, T., Ho, J., Chen, X., Sidor, S., Sutskever, I.: Evolution strategies as a scalable alternative to reinforcement learning. arXiv:1703.03864 – reference: HearnDThe gap function of a convex programOperations Research Letters19822951100486.90070 – reference: DeVoreRPetrovaGWojtaszczykPApproximation of functions of few variables in high dimensionsConstructive Approximation2011331125143274705910.1007/s00365-010-9105-8 – reference: DonohoDLCompressed sensingIEEE Transactions on information theory200652412891306224118910.1109/TIT.2006.871582 – reference: Jin, C., Ge, R., Netrapalli, P., Kakade, S.M., Jordan, M.I.: How to escape saddle points efficiently. In: International Conference on Machine Learning, pp. 1724–1732 (2017) – reference: RioEMoment inequalities for sums of dependent random variables under projective conditionsJournal of Theoretical Probability2009221146163247201010.1007/s10959-008-0155-9 – reference: Beck, A.: First-Order Methods in Optimization, vol. 25. Society for Industrial and Applied Mathematics (SIAM) (2017) – reference: GhadimiSLanGStochastic first- and zeroth-order methods for nonconvex stochastic programmingSIAM Journal on Optimization201323423412368313443910.1137/120880811 – reference: Hazan, E., Levy, K.: Bandit convex optimization: Towards tight bounds. In: Advances in Neural Information Processing Systems, pp. 784–792 (2014) – reference: Cai, H., Mckenzie, D., Yin, W., Zhang, Z.: Zeroth-order regularized optimization (ZORO): Approximately sparse gradients and adaptive sampling (2020) – reference: LanGZhouYConditional gradient sliding for convex optimizationSIAM Journal on Optimization201626213791409351687110.1137/140992382 – reference: MokhtariAHassaniHKarbasiAStochastic conditional gradient methods: From convex minimization to submodular maximizationJournal of Machine Learning Research202021149411917307255136 – reference: Mockus, J.: Bayesian approach to global optimization: theory and applications, vol. 37. Springer Science & Business Media (2012) – reference: Elibol, M., Lei, L., Jordan, M.I.: Variance reduction with sparse gradients. In: Proceedings of the 8th International Conference on Learning Representations (ICLR), pp. 1058–1070 (2020) – reference: Li, J., Balasubramanian, K., Ma, S.: Stochastic zeroth-order riemannian derivative estimation and optimization. arXiv:2003.11238 (2020) – reference: Reddi, S., Zaheer, M., Sra, S., Poczos, B., Bach, F., Salakhutdinov, R., Smola, A.: A generic approach for escaping saddle points. In: International Conference on Artificial Intelligence and Statistics, pp. 1233–1242 (2018) – reference: Wang, Y., Du, S., Balakrishnan, S., Singh, A.: Stochastic zeroth-order optimization in high dimensions. In: A. Storkey, F. Perez-Cruz (eds.) Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, vol. 84, pp. 1356–1365 (2018) – reference: Reddi, S., Sra, S., Póczos, B., Smola, A.: Stochastic Frank-Wolfe Methods for Nonconvex Optimization. In: Proceedings of the 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1244–1251 (2016) – reference: Chen, L., Zhang, M., Hassani, H., Karbasi, A.: Black box submodular maximization: Discrete and continuous settings. In: S. Chiappa, R. Calandra (eds.) Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, vol. 108, pp. 1058–1070 (2020) – reference: Choromanski, K., Rowland, M., Sindhwani, V., Turner, R., Weller, A.: Structured evolution with compact architectures for scalable policy optimization. In: Proceedings of the 35th International Conference on Machine Learning. PMLR (2018) – reference: Shamir, O.: On the complexity of bandit and derivative-free stochastic convex optimization. In: Conference on Learning Theory, pp. 3–24 (2013) – reference: FrankMWolfePAn algorithm for quadratic programmingNaval Research Logistics Quarterly19563951108910210.1002/nav.3800030109 – reference: Chen, P.Y., Zhang, H., Sharma, Y., Yi, J., Hsieh, C.J.: ZOO: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 15–26. ACM (2017) – reference: Kawaguchi, K., Kaelbling, L.P.: Elimination of all bad local minima in deep learning. arXiv:1901.00279 – reference: Mania, H., Guy, A., Recht, B.: Simple random search provides a competitive approach to reinforcement learning. In: Advances in Neural Information Processing Systems (2018) – reference: CartisCGouldNITointPLAdaptive cubic regularisation methods for unconstrained optimization, Part I: Motivation, convergence and numerical resultsMathematical Programming20111272245295277670110.1007/s10107-009-0286-5 – reference: NesterovYPolyakBCubic regularization of newton method and its global performanceMathematical Programming20061081177205222945910.1007/s10107-006-0706-8 – reference: Stein, C.: A bound for the error in the normal approximation to the distribution of a sum of dependent random variables. In: Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Volume 2: Probability Theory. The Regents of the University of California (1972) – reference: Bhojanapalli, S., Neyshabur, B., Srebro, N.: Global optimality of local search for low rank matrix recovery. In: Advances in Neural Information Processing Systems, pp. 3873–3881 (2016) – reference: Akhavan, A., Pontil, M., Tsybakov, A.: Exploiting higher order smoothness in derivative-free optimization and continuous bandits. In: Advances in Neural Information Processing Systems, vol. 33 (2020) – ident: 9499_CR33 – ident: 9499_CR81 – volume: 57 start-page: 101443 year: 2020 ident: 9499_CR36 publication-title: Journal of Complexity doi: 10.1016/j.jco.2019.101443 – volume: 13 start-page: 389 issue: 1 year: 2012 ident: 9499_CR64 publication-title: The Journal of Machine Learning Research – ident: 9499_CR10 – volume-title: Convex optimization year: 2004 ident: 9499_CR11 doi: 10.1017/CBO9780511804441 – ident: 9499_CR6 – ident: 9499_CR66 – volume: 22 start-page: 146 issue: 1 year: 2009 ident: 9499_CR67 publication-title: Journal of Theoretical Probability doi: 10.1007/s10959-008-0155-9 – volume: 27 start-page: 141 issue: 2 year: 2011 ident: 9499_CR82 publication-title: Journal of Complexity doi: 10.1016/j.jco.2011.01.004 – ident: 9499_CR2 – volume: 18 start-page: 1131 issue: 5 year: 2018 ident: 9499_CR77 publication-title: Foundations of Computational Mathematics doi: 10.1007/s10208-017-9365-9 – ident: 9499_CR69 – volume: 26 start-page: 1379 issue: 2 year: 2016 ident: 9499_CR48 publication-title: SIAM Journal on Optimization doi: 10.1137/140992382 – ident: 9499_CR24 – volume: 46 start-page: 2871 issue: 6A year: 2018 ident: 9499_CR52 publication-title: The Annals of Statistics doi: 10.1214/17-AOS1642 – ident: 9499_CR53 – volume-title: Simulation and the Monte Carlo method year: 2016 ident: 9499_CR68 doi: 10.1002/9781118631980 – ident: 9499_CR72 – ident: 9499_CR31 doi: 10.1134/S0005117917020035 – ident: 9499_CR59 – ident: 9499_CR76 – ident: 9499_CR32 – volume: 7 start-page: 183 issue: 2 year: 2018 ident: 9499_CR80 publication-title: Information and Inference: A Journal of the IMA doi: 10.1093/imaiai/iax008 – ident: 9499_CR5 doi: 10.1137/1.9781611974997 – volume: 127 start-page: 245 issue: 2 year: 2011 ident: 9499_CR16 publication-title: Mathematical Programming doi: 10.1007/s10107-009-0286-5 – ident: 9499_CR63 – volume: 17 start-page: 527 year: 2017 ident: 9499_CR61 publication-title: Foundations of Computational Mathematics doi: 10.1007/s10208-015-9296-2 – ident: 9499_CR28 – volume: 130 start-page: 295 issue: 2 year: 2011 ident: 9499_CR17 publication-title: Mathematical Programming doi: 10.1007/s10107-009-0337-y – volume: 184 start-page: 35 year: 2020 ident: 9499_CR83 publication-title: Mathematical Programming doi: 10.1007/s10107-019-01405-z – ident: 9499_CR3 – volume: 52 start-page: 1289 issue: 4 year: 2006 ident: 9499_CR26 publication-title: IEEE Transactions on information theory doi: 10.1109/TIT.2006.871582 – ident: 9499_CR49 – volume-title: Introductory Lectures on Convex Optimization: a basic course year: 2004 ident: 9499_CR58 doi: 10.1007/978-1-4419-8853-9 – ident: 9499_CR79 doi: 10.1007/978-3-319-40519-3_8 – volume: 28 start-page: 1751 issue: 2 year: 2018 ident: 9499_CR15 publication-title: SIAM Journal on Optimization doi: 10.1137/17M1114296 – ident: 9499_CR46 – ident: 9499_CR42 – ident: 9499_CR12 doi: 10.1561/2200000024 – ident: 9499_CR21 – ident: 9499_CR14 – ident: 9499_CR20 doi: 10.1145/3128572.3140448 – ident: 9499_CR65 doi: 10.1109/ALLERTON.2016.7852377 – ident: 9499_CR39 – volume: 23 start-page: 2341 issue: 4 year: 2013 ident: 9499_CR35 publication-title: SIAM Journal on Optimization doi: 10.1137/120880811 – ident: 9499_CR34 doi: 10.1007/s10107-017-1225-5 – volume: 186 start-page: 157 year: 2021 ident: 9499_CR62 publication-title: Mathematical Programming doi: 10.1007/s10107-019-01449-1 – ident: 9499_CR8 – ident: 9499_CR4 – ident: 9499_CR1 – ident: 9499_CR22 doi: 10.1137/1.9780898718768 – volume: 33 start-page: 125 issue: 1 year: 2011 ident: 9499_CR25 publication-title: Constructive Approximation doi: 10.1007/s00365-010-9105-8 – volume: 3 start-page: 95 year: 1956 ident: 9499_CR30 publication-title: Naval Research Logistics Quarterly doi: 10.1002/nav.3800030109 – ident: 9499_CR19 – ident: 9499_CR45 – ident: 9499_CR70 – ident: 9499_CR41 – volume-title: Problem complexity and method efficiency in optimization year: 1983 ident: 9499_CR57 – ident: 9499_CR38 – ident: 9499_CR43 doi: 10.1561/2200000058 – ident: 9499_CR51 – ident: 9499_CR78 – ident: 9499_CR13 doi: 10.1145/3055399.3055403 – ident: 9499_CR74 – volume: 18 start-page: 1073 issue: 5 year: 2018 ident: 9499_CR18 publication-title: Foundations of Computational Mathematics doi: 10.1007/s10208-017-9363-y – ident: 9499_CR7 doi: 10.1137/1.9780898718829 – volume: 2 start-page: 95 year: 1982 ident: 9499_CR40 publication-title: Operations Research Letters – ident: 9499_CR9 – volume: 21 start-page: 1 year: 2020 ident: 9499_CR55 publication-title: Journal of Machine Learning Research – ident: 9499_CR75 doi: 10.1214/aos/1176345632 – ident: 9499_CR47 – volume: 39 start-page: 117 issue: 2 year: 1987 ident: 9499_CR56 publication-title: Mathematical Programming doi: 10.1007/BF02592948 – volume: 108 start-page: 177 issue: 1 year: 2006 ident: 9499_CR60 publication-title: Mathematical Programming doi: 10.1007/s10107-006-0706-8 – ident: 9499_CR71 – volume-title: Introduction to stochastic search and optimization: estimation, simulation, and control year: 2005 ident: 9499_CR73 – ident: 9499_CR44 – ident: 9499_CR23 – volume: 17 start-page: 7565 issue: 1 year: 2016 ident: 9499_CR29 publication-title: The Journal of Machine Learning Research – volume: 61 start-page: 2788 issue: 5 year: 2015 ident: 9499_CR27 publication-title: IEEE Transactions on Information Theory doi: 10.1109/TIT.2015.2409256 – ident: 9499_CR37 – ident: 9499_CR50 – ident: 9499_CR54
SSID	ssj0015914 ssib031263371
Score	2.5678225
Snippet	In this paper, we propose and analyze zeroth-order stochastic approximation algorithms for nonconvex and convex optimization, with a focus on addressing...
SourceID	proquest crossref springer
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	35
SubjectTerms	Algorithms Applications of Mathematics Computational geometry Computer Science Constraints Convergence Convexity Economics Hessian matrices Image processing Linear and Multilinear Algebras Math Applications in Computer Science Mathematical analysis Mathematics Mathematics and Statistics Matrix Theory Newton methods Numerical Analysis Optimization Regularization Saddle points Smoothing Sparsity
Title	Zeroth-Order Nonconvex Stochastic Optimization: Handling Constraints, High Dimensionality, and Saddle Points
URI	https://link.springer.com/article/10.1007/s10208-021-09499-8 https://www.proquest.com/docview/2625578754
Volume	22
WOSCitedRecordID	wos000630642400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVAVX databaseName: SpringerLINK Contemporary 1997-Present customDbUrl: eissn: 1615-3383 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0015914 issn: 1615-3375 databaseCode: RSV dateStart: 20010101 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3fT9swED4xmLTtYbBuE4WC_MDbailx7NjZ2wSreIAOqYCqvUSx44pJLJmagPjz53OddiCYBFKenIvl-Mf5s-_uO4CDLObaoQpNjdOElDNZUIcyNE2MSFVqY6t96oTLEzkeq-k0OwtBYU3n7d6ZJL2m_ifYjaGpnrnjLzKqUPUKNgSyzeAZfXK5tB2IzDN6I5ShSSJFCJV5vI7729EKYz4wi_rdZrT5snZuwfuALsm3xXT4AGu26sFmQJokrOPGFXXJHLqyHrzpQpTd63enSzLX5iNc_7TzGm9AkaWTjOvKe6rfkUlbm6sCeZ7JD6d4foeIzq_kGJkbXJMJZgP1OSjaZkjQo4QcYTKBBRGIg_9D4iTJpMA7EnJWo9wnuBh9Pz88piFJAzWc8ZZKEyutElVwt8-xLMWwrpIx90iEkmUiSyZUOpup0m3HRiNhv9ZRqoSUThvMks-wXtWV3QZiWSlTHhXcCLQfC51FlscZT7WJkqKwfYi7scpNYDDHn7jOV9zL2Pe56_vc932u-vBl-c2fBX_Hf6UH3RTIw1pucuaOiKjXBO_DsBvy1euna9t5nvguvGUYW-Fdwgew3s5v7B68Nrftr2a-7-f4X-7x88g
linkProvider	Springer Nature
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3fT9swED5tHVLhYQUGWkc3_MDbaqlxnNjhbWKrOq0tSAVU7SWKHVcglWZqsok_H5_rtBuCSZuUJ-diOf5x_uy7-w7gJAm4sqhCUW01IeVMZNSiDEVDHcUyNoFRLnXC9VCMx3I6TS58UFhZe7vXJkmnqX8LdmNoqmf2-IuMKlS-hFcc0-zgGX1yvbYdRIlj9EYoQ8NQRD5U5uk6_tyONhjzkVnU7Tb91v-1cxdee3RJPq2mwx68MIt9aHmkSfw6Lm1RncyhLtuHZh2ibF_vjNZkruUbmH83ywJvQJGlk4yLhfNUvyeTqtA3GfI8k3OreO58ROcpGSBzg20ywWygLgdFVXYJepSQz5hMYEUEYuF_l1hJMsnwjoRcFCh3AFf9L5dnA-qTNFDNGa-o0IFUMpQZt_scS2IM68oZs49AKJmHImeRjGczmdvtWCsk7FeqF8tICKsNZuEhNBbFwrwFYlguYt7LuI7QfhyppGd4kPBY6V6YZaYNQT1WqfYM5vgT83TDvYx9n9q-T13fp7INH9ff_Fjxd_xVulNPgdSv5TJl9oiIei3ibejWQ755_Xxt7_5N_Biag8vRMB1-HX87gm2GcRbOPbwDjWr507yHLf2rui2XH9x8fwBjtvas
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3db9MwED-xMm3wwNgYojDAD3tbrTWOEzu8IUo1xCjVOqZpL1HsuGLSlkxNmPbnc-cm7YYGEpqUJ-diOf64D9_d7wB2k0Aa1CoMt8gJuRQq46hlGB7aKNaxC5zxpRNODtVopE9Pk_GtLH4f7d66JOc5DYTSVNT7V_l0_1bimyC3vUBTmNBVuF6BxxItGQrqOpqcLPwIUeLRvUmt4WGooiZt5v4-7oqmpb75h4vUS57hxsPH_ByeNVon-zjfJpvwyBVbsNFooKw53xU2tUUe2rYtWG9Tl_H1028LkNfqBVycuVlJN6OE3slGZeEj2G_YpC7tz4zwn9l3ZEiXTabnB3ZAiA44fEZVQn1tirrqMYo0YQMqMjAHCEGzoMeQkk0yujth45LotuHH8PPxpwPeFG_gVgpZc2UDbXSoM4nyTyQxpXvlQuCjSMXMQ5WLSMfTqc5RTFtDQP7G9GMdKYVcYhq-hE5RFu4VMCdyFct-Jm1EfuXIJH0ng0TGxvbDLHNdCNp1S22DbE4_cZEuMZlp7lOc-9TPfaq7sLf45mqO6_FP6p12O6TNGa9SgaYj8btIdqHXLv_y9d97e_1_5O9hbTwYpodfRl_fwBNB6Rc-anwHOvXsl3sLq_a6Pq9m7_zW_w1P7_-Q
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Zeroth-Order+Nonconvex+Stochastic+Optimization%3A+Handling+Constraints%2C+High+Dimensionality%2C+and+Saddle+Points&rft.jtitle=Foundations+of+computational+mathematics&rft.au=Balasubramanian+Krishnakumar&rft.au=Ghadimi+Saeed&rft.date=2022-02-01&rft.pub=Springer+Nature+B.V&rft.issn=1615-3375&rft.eissn=1615-3383&rft.volume=22&rft.issue=1&rft.spage=35&rft.epage=76&rft_id=info:doi/10.1007%2Fs10208-021-09499-8&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1615-3375&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1615-3375&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1615-3375&client=summon