A scaled BFGS preconditioned conjugate gradient algorithm for unconstrained optimization

This letter presents a scaled memoryless BFGS preconditioned conjugate gradient algorithm for solving unconstrained optimization problems. The basic idea is to combine the scaled memoryless BFGS method and the preconditioning technique in the frame of the conjugate gradient method. The preconditione...

Full description

Saved in:

Bibliographic Details
Published in:	Applied mathematics letters Vol. 20; no. 6; pp. 645 - 650
Main Author:	Andrei, Neculai
Format:	Journal Article
Language:	English
Published:	Oxford Elsevier Ltd 01.06.2007 Elsevier
Subjects:	BFGS preconditioning Conjugate gradient method Exact sciences and technology Mathematical analysis Mathematics Numerical analysis Numerical analysis. Scientific computation Numerical linear algebra Sciences and techniques of general use Unconstrained optimization Conjugate gradient method Unconstrained optimization BFGS preconditioning Frame Gradient Spectral method Program Memory Optimization method Algorithm Direction Convergence Letter Applied mathematics Problem solving Metric Large scale Limit Preconditioning
ISSN:	0893-9659, 1873-5452
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	This letter presents a scaled memoryless BFGS preconditioned conjugate gradient algorithm for solving unconstrained optimization problems. The basic idea is to combine the scaled memoryless BFGS method and the preconditioning technique in the frame of the conjugate gradient method. The preconditioner, which is also a scaled memoryless BFGS matrix, is reset when the Powell restart criterion holds. The parameter scaling the gradient is selected as the spectral gradient. Computational results for a set consisting of 750 test unconstrained optimization problems show that this new scaled conjugate gradient algorithm substantially outperforms known conjugate gradient methods such as the spectral conjugate gradient SCG of Birgin and Martínez [E. Birgin, J.M. Martínez, A spectral conjugate gradient method for unconstrained optimization, Appl. Math. Optim. 43 (2001) 117–128] and the (classical) conjugate gradient of Polak and Ribière [E. Polak, G. Ribière, Note sur la convergence de méthodes de directions conjuguées, Revue Francaise Informat. Reserche Opérationnelle, 3e Année 16 (1969) 35–43], but subject to the CPU time metric it is outperformed by L-BFGS [D. Liu, J. Nocedal, On the limited memory BFGS method for large scale optimization, Math. Program. B 45 (1989) 503–528; J. Nocedal. http://www.ece.northwestern.edu/~nocedal/lbfgs.html].
AbstractList	This letter presents a scaled memoryless BFGS preconditioned conjugate gradient algorithm for solving unconstrained optimization problems. The basic idea is to combine the scaled memoryless BFGS method and the preconditioning technique in the frame of the conjugate gradient method. The preconditioner, which is also a scaled memoryless BFGS matrix, is reset when the Powell restart criterion holds. The parameter scaling the gradient is selected as the spectral gradient. Computational results for a set consisting of 750 test unconstrained optimization problems show that this new scaled conjugate gradient algorithm substantially outperforms known conjugate gradient methods such as the spectral conjugate gradient SCG of Birgin and Martínez [E. Birgin, J.M. Martínez, A spectral conjugate gradient method for unconstrained optimization, Appl. Math. Optim. 43 (2001) 117–128] and the (classical) conjugate gradient of Polak and Ribière [E. Polak, G. Ribière, Note sur la convergence de méthodes de directions conjuguées, Revue Francaise Informat. Reserche Opérationnelle, 3e Année 16 (1969) 35–43], but subject to the CPU time metric it is outperformed by L-BFGS [D. Liu, J. Nocedal, On the limited memory BFGS method for large scale optimization, Math. Program. B 45 (1989) 503–528; J. Nocedal. http://www.ece.northwestern.edu/~nocedal/lbfgs.html].
Author	Andrei, Neculai
Author_xml	– sequence: 1 givenname: Neculai surname: Andrei fullname: Andrei, Neculai email: nandrei@ici.ro organization: Research Institute for Informatics, Center for Advanced Modeling and Optimization, 8-10, Averescu Avenue, Bucharest 1, Romania
BackLink	http://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&idt=18658063$$DView record in Pascal Francis
BookMark	eNp9kM1KAzEUhYMo2FYfwN1sXM6YTJpMgqtabBUKLlRwF9IkUzPMH0kq6NObWEFwUbhwb8j5DpwzBaf90BsArhAsEET0pilk1xYlhLRIg8gJmCBW4ZzMSXkKJpBxnHNK-DmYet9ACDHHbALeFplXsjU6u1utn7PRGTX02gYb7XUW72a_k8FkOye1NX3IZLsbnA3vXVYPLtv3UeKDkzbJhzHYzn7JRF-As1q23lz-7hl4Xd2_LB_yzdP6cbnY5AqXVcglNnSLtxhSLOcwvqBGlBmK5xqVEjNj8FZyTWBVMS4JQbSCHJdEGw7jp8YzcH3wHWUKUjvZK-vF6Gwn3adAjBIWzaMOHXTKDd47U_9JoEgVikbECkWqUKRBJDLVP0bZ8JMuJW6PkrcH0sToH9Y44VWsTxltY8NB6MEeob8BCH2OTA
CitedBy_id	crossref_primary_10_1007_s10957_019_01527_6 crossref_primary_10_1007_s11012_019_01006_2 crossref_primary_10_3390_nano15161289 crossref_primary_10_1007_s10957_008_9505_0 crossref_primary_10_1007_s10589_013_9558_3 crossref_primary_10_1155_2023_9945581 crossref_primary_10_1016_j_ejor_2009_11_030 crossref_primary_10_1155_2015_961487 crossref_primary_10_1007_s11770_020_0844_4 crossref_primary_10_1007_s10288_013_0233_4 crossref_primary_10_1007_s10957_015_0724_x crossref_primary_10_1155_2021_9919595 crossref_primary_10_1002_eng2_12968 crossref_primary_10_1080_03610926_2012_661510 crossref_primary_10_1016_j_jmgm_2016_04_002 crossref_primary_10_1007_s10589_023_00508_w crossref_primary_10_1016_j_cmpb_2017_04_012 crossref_primary_10_1016_j_matcom_2023_07_026 crossref_primary_10_1016_j_camwa_2018_10_002 crossref_primary_10_1007_s11665_015_1383_7 crossref_primary_10_1007_s11590_012_0462_z crossref_primary_10_1007_s11075_015_0053_z crossref_primary_10_1080_10556788_2024_2372668 crossref_primary_10_1007_s11075_007_9152_9 crossref_primary_10_1155_2013_286486 crossref_primary_10_1080_02331934_2025_2547719 crossref_primary_10_1016_j_aml_2010_01_010 crossref_primary_10_1016_j_cam_2013_11_001 crossref_primary_10_1007_s11075_019_00658_1 crossref_primary_10_1016_j_cam_2021_113582 crossref_primary_10_1080_0305215X_2025_2475004 crossref_primary_10_1007_s40995_020_01012_0 crossref_primary_10_1080_02522667_2009_10699885 crossref_primary_10_1109_ACCESS_2021_3081570 crossref_primary_10_1007_s10288_020_00432_3 crossref_primary_10_1080_10556788_2018_1457152 crossref_primary_10_1016_j_amc_2009_03_020 crossref_primary_10_1155_2013_517452 crossref_primary_10_1016_j_cam_2022_114630 crossref_primary_10_1080_10556788_2016_1225213 crossref_primary_10_1007_s12190_022_01713_2 crossref_primary_10_1016_j_cam_2010_01_052 crossref_primary_10_1080_10556788_2010_501379 crossref_primary_10_1016_j_cam_2008_12_024 crossref_primary_10_1108_09615531311301272
Cites_doi	10.1137/1013035 10.1287/moor.3.3.244 10.1137/0715085 10.1137/1011036 10.1145/200979.201043 10.1007/s002450010019 10.1007/s00245-001-0003-0 10.1093/comjnl/7.2.149 10.6028/jres.049.044 10.1007/BF01593790 10.1007/BF01589116
ContentType	Journal Article
Copyright	2006 Elsevier Ltd 2007 INIST-CNRS
Copyright_xml	– notice: 2006 Elsevier Ltd – notice: 2007 INIST-CNRS
DBID	6I. AAFTH AAYXX CITATION IQODW
DOI	10.1016/j.aml.2006.06.015
DatabaseName	ScienceDirect Open Access Titles Elsevier:ScienceDirect:Open Access CrossRef Pascal-Francis
DatabaseTitle	CrossRef
DatabaseTitleList
DeliveryMethod	fulltext_linktorsrc
Discipline	Mathematics
EISSN	1873-5452
EndPage	650
ExternalDocumentID	18658063 10_1016_j_aml_2006_06_015 S0893965906002382
GroupedDBID	--K --M .~1 0R~ 1B1 1RT 1~. 1~5 23M 4.4 457 4G. 5GY 5VS 6I. 7-5 71M 8P~ 9JN AACTN AAEDT AAEDW AAFTH AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXUO ABAOU ABEFU ABFNM ABJNI ABMAC ABVKL ABXDB ABYKQ ACAZW ACDAQ ACGFS ACNNM ACRLP ADBBV ADEZE ADMUD ADTZH AEBSH AECPX AEKER AENEX AEXQZ AFFNX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHJVU AIEXJ AIGVJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ ARUGR ASPBG AVWKF AXJTR AZFZN BJAXD BKOJK BLXMC CS3 EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN G-2 G-Q GBLVA HVGLF HZ~ IHE IXB J1W JJJVA KOM M26 M41 MHUIS MO0 N9A NCXOZ O-L O9- OAUVE OK1 OZT P-8 P-9 P2P PC. Q38 R2- RIG RNS ROL RPZ SDF SDG SDP SES SEW SPC SPCBC SST SSW SSZ T5K UHS WUQ XPP ZMT ~G- 9DU AATTM AAXKI AAYWO AAYXX ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO ADVLN AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP CITATION EFKBS ~HD AFXIZ AGCQF AGRNS BNPGV IQODW SSH
ID	FETCH-LOGICAL-c327t-a3e6b3b3063a40a3e0d168e634d12a38ee3ba9d507789a5516709325de908eed3
ISICitedReferencesCount	69
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000246062200008&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0893-9659
IngestDate	Mon Jul 21 09:15:37 EDT 2025 Sat Nov 29 07:11:20 EST 2025 Tue Nov 18 21:00:26 EST 2025 Fri Feb 23 02:17:35 EST 2024
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	6
Keywords	Conjugate gradient method Unconstrained optimization BFGS preconditioning Frame Gradient Spectral method Program Memory Optimization method Algorithm Direction Convergence Letter Applied mathematics Problem solving Metric Large scale Limit Preconditioning
Language	English
License	http://www.elsevier.com/open-access/userlicense/1.0 CC BY 4.0
LinkModel	OpenURL
MergedId	FETCHMERGED-LOGICAL-c327t-a3e6b3b3063a40a3e0d168e634d12a38ee3ba9d507789a5516709325de908eed3
OpenAccessLink	https://dx.doi.org/10.1016/j.aml.2006.06.015
PageCount	6
ParticipantIDs	pascalfrancis_primary_18658063 crossref_primary_10_1016_j_aml_2006_06_015 crossref_citationtrail_10_1016_j_aml_2006_06_015 elsevier_sciencedirect_doi_10_1016_j_aml_2006_06_015
PublicationCentury	2000
PublicationDate	2007-06-01
PublicationDateYYYYMMDD	2007-06-01
PublicationDate_xml	– month: 06 year: 2007 text: 2007-06-01 day: 01
PublicationDecade	2000
PublicationPlace	Oxford
PublicationPlace_xml	– name: Oxford
PublicationTitle	Applied mathematics letters
PublicationYear	2007
Publisher	Elsevier Ltd Elsevier
Publisher_xml	– name: Elsevier Ltd – name: Elsevier
References	Nocedal (b7) Wolfe (b14) 1971; 13 Powell (b10) 1977; 12 Shanno (b11) 1978; 3 Birgin, Martínez (b1) 2001; 43 Polak, Ribière (b9) 1969; 16 Fletcher, Reeves (b4) 1964; 7 Shanno (b12) 1978; 15 J.M. Perry, A class of conjugate gradient algorithms with a two step variable metric memory, Discussion paper 269, Center for Mathematical Studies in Economics and Management Science, Northwestern University, 1977 Wolfe (b13) 1969; 11 Hestenes, Stiefel (b5) 1952; 48 Bongartz, Conn, Gould, Toint (b2) 1995; 21 Dai, Liao (b3) 2001; 43 Liu, Nocedal (b6) 1989; 45 Birgin (10.1016/j.aml.2006.06.015_b1) 2001; 43 10.1016/j.aml.2006.06.015_b8 Wolfe (10.1016/j.aml.2006.06.015_b13) 1969; 11 Fletcher (10.1016/j.aml.2006.06.015_b4) 1964; 7 Shanno (10.1016/j.aml.2006.06.015_b11) 1978; 3 Wolfe (10.1016/j.aml.2006.06.015_b14) 1971; 13 Polak (10.1016/j.aml.2006.06.015_b9) 1969; 16 Liu (10.1016/j.aml.2006.06.015_b6) 1989; 45 Hestenes (10.1016/j.aml.2006.06.015_b5) 1952; 48 Bongartz (10.1016/j.aml.2006.06.015_b2) 1995; 21 Dai (10.1016/j.aml.2006.06.015_b3) 2001; 43 Powell (10.1016/j.aml.2006.06.015_b10) 1977; 12 Nocedal (10.1016/j.aml.2006.06.015_b7) Shanno (10.1016/j.aml.2006.06.015_b12) 1978; 15
References_xml	– volume: 3 start-page: 244 year: 1978 end-page: 256 ident: b11 article-title: Conjugate gradient methods with inexact searches publication-title: Math. Oper. Res. – volume: 15 start-page: 1247 year: 1978 end-page: 1257 ident: b12 article-title: On the convergence of a new conjugate gradient algorithm publication-title: SIAM J. Numer. Anal. – volume: 45 start-page: 503 year: 1989 end-page: 528 ident: b6 article-title: On the limited memory BFGS method for large scale optimization publication-title: Math. Program. B – volume: 11 start-page: 226 year: 1969 end-page: 235 ident: b13 article-title: Convergence conditions for ascent methods publication-title: SIAM Rev. – volume: 7 start-page: 149 year: 1964 end-page: 154 ident: b4 article-title: Function minimization by conjugate gradients publication-title: Comput. J. – volume: 48 start-page: 409 year: 1952 end-page: 436 ident: b5 article-title: Methods of conjugate gradients for solving linear systems publication-title: J. Res. Nat. Bur. Stan. Sec. B – volume: 43 start-page: 117 year: 2001 end-page: 128 ident: b1 article-title: A spectral conjugate gradient method for unconstrained optimization publication-title: Appl. Math. Optim. – volume: 21 start-page: 123 year: 1995 end-page: 160 ident: b2 article-title: CUTE: Constrained and unconstrained testing environments publication-title: ACM Trans. Math. Software – ident: b7 – reference: J.M. Perry, A class of conjugate gradient algorithms with a two step variable metric memory, Discussion paper 269, Center for Mathematical Studies in Economics and Management Science, Northwestern University, 1977 – volume: 43 start-page: 87 year: 2001 end-page: 101 ident: b3 article-title: New conjugate conditions and related nonlinear conjugate gradient methods publication-title: Appl. Math. Optim. – volume: 16 start-page: 35 year: 1969 end-page: 43 ident: b9 article-title: Note sur la convergence de méthodes de directions conjuguées publication-title: Revue Francaise Informat. Reserche Opérationnelle, 3e Année – volume: 12 start-page: 241 year: 1977 end-page: 254 ident: b10 article-title: Restart procedures for the conjugate gradient method publication-title: Math. Program. – volume: 13 start-page: 185 year: 1971 end-page: 188 ident: b14 article-title: Convergence conditions for ascent methods II: Some corrections publication-title: SIAM Rev. – volume: 13 start-page: 185 year: 1971 ident: 10.1016/j.aml.2006.06.015_b14 article-title: Convergence conditions for ascent methods II: Some corrections publication-title: SIAM Rev. doi: 10.1137/1013035 – ident: 10.1016/j.aml.2006.06.015_b7 – volume: 3 start-page: 244 year: 1978 ident: 10.1016/j.aml.2006.06.015_b11 article-title: Conjugate gradient methods with inexact searches publication-title: Math. Oper. Res. doi: 10.1287/moor.3.3.244 – volume: 16 start-page: 35 year: 1969 ident: 10.1016/j.aml.2006.06.015_b9 article-title: Note sur la convergence de méthodes de directions conjuguées publication-title: Revue Francaise Informat. Reserche Opérationnelle, 3e Année – volume: 15 start-page: 1247 year: 1978 ident: 10.1016/j.aml.2006.06.015_b12 article-title: On the convergence of a new conjugate gradient algorithm publication-title: SIAM J. Numer. Anal. doi: 10.1137/0715085 – volume: 11 start-page: 226 year: 1969 ident: 10.1016/j.aml.2006.06.015_b13 article-title: Convergence conditions for ascent methods publication-title: SIAM Rev. doi: 10.1137/1011036 – volume: 21 start-page: 123 year: 1995 ident: 10.1016/j.aml.2006.06.015_b2 article-title: CUTE: Constrained and unconstrained testing environments publication-title: ACM Trans. Math. Software doi: 10.1145/200979.201043 – volume: 43 start-page: 87 year: 2001 ident: 10.1016/j.aml.2006.06.015_b3 article-title: New conjugate conditions and related nonlinear conjugate gradient methods publication-title: Appl. Math. Optim. doi: 10.1007/s002450010019 – ident: 10.1016/j.aml.2006.06.015_b8 – volume: 43 start-page: 117 year: 2001 ident: 10.1016/j.aml.2006.06.015_b1 article-title: A spectral conjugate gradient method for unconstrained optimization publication-title: Appl. Math. Optim. doi: 10.1007/s00245-001-0003-0 – volume: 7 start-page: 149 year: 1964 ident: 10.1016/j.aml.2006.06.015_b4 article-title: Function minimization by conjugate gradients publication-title: Comput. J. doi: 10.1093/comjnl/7.2.149 – volume: 48 start-page: 409 year: 1952 ident: 10.1016/j.aml.2006.06.015_b5 article-title: Methods of conjugate gradients for solving linear systems publication-title: J. Res. Nat. Bur. Stan. Sec. B doi: 10.6028/jres.049.044 – volume: 12 start-page: 241 year: 1977 ident: 10.1016/j.aml.2006.06.015_b10 article-title: Restart procedures for the conjugate gradient method publication-title: Math. Program. doi: 10.1007/BF01593790 – volume: 45 start-page: 503 year: 1989 ident: 10.1016/j.aml.2006.06.015_b6 article-title: On the limited memory BFGS method for large scale optimization publication-title: Math. Program. B doi: 10.1007/BF01589116
SSID	ssj0003938
Score	2.089182
Snippet	This letter presents a scaled memoryless BFGS preconditioned conjugate gradient algorithm for solving unconstrained optimization problems. The basic idea is to...
SourceID	pascalfrancis crossref elsevier
SourceType	Index Database Enrichment Source Publisher
StartPage	645
SubjectTerms	BFGS preconditioning Conjugate gradient method Exact sciences and technology Mathematical analysis Mathematics Numerical analysis Numerical analysis. Scientific computation Numerical linear algebra Sciences and techniques of general use Unconstrained optimization
Title	A scaled BFGS preconditioned conjugate gradient algorithm for unconstrained optimization
URI	https://dx.doi.org/10.1016/j.aml.2006.06.015
Volume	20
WOSCitedRecordID	wos000246062200008&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1873-5452 dateEnd: 20180131 omitProxy: false ssIdentifier: ssj0003938 issn: 0893-9659 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3db9MwELdQxwMITXxqYzD5gSeqoMRO7PixoI2BRIW0IfUtcmN3tGqzqu3Q_nzubOcLtAmQkJIoSfPh-n65-9k-3xHyxvAMrV4aAXx0lDLDIsWsibSVcgoWLJ2J0iWbkONxPpmoryFd6talE5BVld_cqPV_FTWcA2Hj1Nm_EHfzUDgB-yB02ILYYftHgh8Nt1DvwCPfn348xxgA0OI1PiIRzmCrFtfYcza83DhnL-zxvbzazHffV87jEMwcMkZMHAGXX4FCWYWZml0aW3PXVRP0dTtcuolBDUV3npLOVWDs8u_Oe_0LsvWD8p1e9cSX1svI6SbFI4xF6M2I1525RD-LrKdcWdwBUVdTCh9FMhhd4aPP_qbPfdfC4p1eLcPAESxJ1hqvxqXwHIuEJYqF4yFglveYzFQ-IHujTyeTz4195srlN2_-Qj3W7bz-fnnRbWzl0VqjLGc--UmHkVw8JvuhKUFHHgJPyD1bPSUPv7QieUYmI-rBQBEMtA8G2oCB1mCgDRgogIH2wEC7YHhOvp2eXHw4i0IujajkTO4iza2Y8ik0ELlOYziKTSJyK3hqEqZ5bi2famWgdSBzpXH0VMZA7TNjVQw_Gv6CDCoo2wGh1kCrkpczLiyHnUSXqpScS1hjyUp2SOK6zooyBJrHki6L2qNwUUA1YwJUUeCSZIfkbXPL2kdZuevitBZEEWiip38FoOau2457QmtflAMLh3p5-W_PPSIP2s_mFRnsNtf2Nblf_tjNt5vjAL6fNQOXtQ
linkProvider	Elsevier
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+scaled+BFGS+preconditioned+conjugate+gradient+algorithm+for+unconstrained+optimization&rft.jtitle=Applied+mathematics+letters&rft.au=Andrei%2C+Neculai&rft.date=2007-06-01&rft.pub=Elsevier+Ltd&rft.issn=0893-9659&rft.eissn=1873-5452&rft.volume=20&rft.issue=6&rft.spage=645&rft.epage=650&rft_id=info:doi/10.1016%2Fj.aml.2006.06.015&rft.externalDocID=S0893965906002382
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0893-9659&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0893-9659&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0893-9659&client=summon