Scalable Primal Heuristics Using Graph Neural Networks for Combinatorial Optimization

By examining the patterns of solutions obtained for various instances, one can gain insights into the structure and behavior of combinatorial optimization (CO) problems and develop efficient algorithms for solving them. Machine learning techniques, especially Graph Neural Networks (GNNs), have shown...

Full description

Saved in:
Bibliographic Details
Published in:The Journal of artificial intelligence research Vol. 80; pp. 327 - 376
Main Authors: Cantürk, Furkan, Varol, Taha, Aydoğan, Reyhan, Özener, Okan Örsan
Format: Journal Article
Language:English
Published: 01.01.2024
ISSN:1076-9757, 1076-9757
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract By examining the patterns of solutions obtained for various instances, one can gain insights into the structure and behavior of combinatorial optimization (CO) problems and develop efficient algorithms for solving them. Machine learning techniques, especially Graph Neural Networks (GNNs), have shown promise in parametrizing and automating this laborious design process. The inductive bias of GNNs allows for learning solutions to mixed-integer programming (MIP) formulations of constrained CO problems with a relational representation of decision variables and constraints. The trained GNNs can be leveraged with primal heuristics to construct high-quality feasible solutions to CO problems quickly. However, current GNN-based end-to-end learning approaches have limitations for scalable training and generalization on larger-scale instances; therefore, they have been mostly evaluated over small-scale instances. Addressing this issue, our study builds on supervised learning of optimal solutions to the downscaled instances of given large-scale CO problems. We introduce several improvements on a recent GNN model for CO to generalize on instances of a larger scale than those used in training. We also propose a two-stage primal heuristic strategy based on uncertainty-quantification to automatically configure how solution search relies on the predicted decision values. Our models can generalize on 16x upscaled instances of commonly benchmarked five CO problems. Unlike the regressive performance of existing GNN-based CO approaches as the scale of problems increases, the CO pipelines using our models offer an incremental performance improvement relative to CPLEX. The proposed uncertainty-based primal heuristics provide 6-75% better optimality gap values and 45-99% better primal gap values for the 16x upscaled instances and brings immense speedup to obtain high-quality solutions. All these gains are achieved through a computationally efficient modeling approach without sacrificing solution quality.
AbstractList By examining the patterns of solutions obtained for various instances, one can gain insights into the structure and behavior of combinatorial optimization (CO) problems and develop efficient algorithms for solving them. Machine learning techniques, especially Graph Neural Networks (GNNs), have shown promise in parametrizing and automating this laborious design process. The inductive bias of GNNs allows for learning solutions to mixed-integer programming (MIP) formulations of constrained CO problems with a relational representation of decision variables and constraints. The trained GNNs can be leveraged with primal heuristics to construct high-quality feasible solutions to CO problems quickly. However, current GNN-based end-to-end learning approaches have limitations for scalable training and generalization on larger-scale instances; therefore, they have been mostly evaluated over small-scale instances. Addressing this issue, our study builds on supervised learning of optimal solutions to the downscaled instances of given large-scale CO problems. We introduce several improvements on a recent GNN model for CO to generalize on instances of a larger scale than those used in training. We also propose a two-stage primal heuristic strategy based on uncertainty-quantification to automatically configure how solution search relies on the predicted decision values. Our models can generalize on 16x upscaled instances of commonly benchmarked five CO problems. Unlike the regressive performance of existing GNN-based CO approaches as the scale of problems increases, the CO pipelines using our models offer an incremental performance improvement relative to CPLEX. The proposed uncertainty-based primal heuristics provide 6-75% better optimality gap values and 45-99% better primal gap values for the 16x upscaled instances and brings immense speedup to obtain high-quality solutions. All these gains are achieved through a computationally efficient modeling approach without sacrificing solution quality.
Author Varol, Taha
Aydoğan, Reyhan
Özener, Okan Örsan
Cantürk, Furkan
Author_xml – sequence: 1
  givenname: Furkan
  orcidid: 0000-0003-4937-6538
  surname: Cantürk
  fullname: Cantürk, Furkan
– sequence: 2
  givenname: Taha
  orcidid: 0000-0001-8831-2700
  surname: Varol
  fullname: Varol, Taha
– sequence: 3
  givenname: Reyhan
  orcidid: 0000-0002-5260-9999
  surname: Aydoğan
  fullname: Aydoğan, Reyhan
– sequence: 4
  givenname: Okan Örsan
  orcidid: 0000-0002-9291-1877
  surname: Özener
  fullname: Özener, Okan Örsan
BookMark eNpNkFFLwzAcxINMcJu--QHyAWzNP2mT9lGKbsLYBO1zSdNUM9umJBHRT79WffDpjjs4uN8KLQY7aISugcTAgd0epXExxJDkgp6hJRDBo1ykYvHPX6CV90dCIE9otkTls5KdrDuNn5zpZYe3-sMZH4zyuPRmeMUbJ8c3vJ_iqd3r8Gndu8etdbiwfW0GGawzU3UYg-nNtwzGDpfovJWd11d_ukblw_1LsY12h81jcbeLFGUkREwLIqjMWwKE1UowpqUiIiOcUtmklCmV1cBqaFLNkzZnNdEAbco58CRvgK3Rze-uctZ7p9tqnF-4rwpINSOpZiQVVD9I2AllTVc-
ContentType Journal Article
DBID AAYXX
CITATION
DOI 10.1613/jair.1.14972
DatabaseName CrossRef
DatabaseTitle CrossRef
DatabaseTitleList CrossRef
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 1076-9757
EndPage 376
ExternalDocumentID 10_1613_jair_1_14972
GroupedDBID .DC
29J
2WC
5GY
5VS
AAKMM
AAKPC
AALFJ
AAYFX
AAYXX
ACGFO
ACM
ADBBV
ADBSK
ADMLS
AEFXT
AEJOY
AENEX
AFFHD
AFKRA
AFWXC
AKRVB
ALMA_UNASSIGNED_HOLDINGS
AMVHM
ARAPS
BCNDV
BENPR
BGLVJ
CCPQU
CITATION
E3Z
EBS
EJD
F5P
FRJ
FRP
GROUPED_DOAJ
GUFHI
HCIFZ
K7-
KQ8
LHSKQ
LPJ
OK1
OVT
P2P
PHGZM
PHGZT
PIMPY
PQGLB
RNS
TR2
XSB
ID FETCH-LOGICAL-c230t-3e7072a9f0103bc733eac0780622ad523cc8b13b1d5e64f93b0e11f5661649d13
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001243395300002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1076-9757
IngestDate Sat Nov 29 05:27:07 EST 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Language English
LinkModel OpenURL
MergedId FETCHMERGED-LOGICAL-c230t-3e7072a9f0103bc733eac0780622ad523cc8b13b1d5e64f93b0e11f5661649d13
ORCID 0000-0001-8831-2700
0000-0002-9291-1877
0000-0002-5260-9999
0000-0003-4937-6538
OpenAccessLink https://jair.org/index.php/jair/article/download/14972/27041
PageCount 50
ParticipantIDs crossref_primary_10_1613_jair_1_14972
PublicationCentury 2000
PublicationDate 2024-01-01
PublicationDateYYYYMMDD 2024-01-01
PublicationDate_xml – month: 01
  year: 2024
  text: 2024-01-01
  day: 01
PublicationDecade 2020
PublicationTitle The Journal of artificial intelligence research
PublicationYear 2024
SSID ssj0019428
Score 2.4178593
Snippet By examining the patterns of solutions obtained for various instances, one can gain insights into the structure and behavior of combinatorial optimization (CO)...
SourceID crossref
SourceType Index Database
StartPage 327
Title Scalable Primal Heuristics Using Graph Neural Networks for Combinatorial Optimization
Volume 80
WOSCitedRecordID wos001243395300002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVAON
  databaseName: DOAJ Directory of Open Access Journals
  customDbUrl:
  eissn: 1076-9757
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0019428
  issn: 1076-9757
  databaseCode: DOA
  dateStart: 19930101
  isFulltext: true
  titleUrlDefault: https://www.doaj.org/
  providerName: Directory of Open Access Journals
– providerCode: PRVPQU
  databaseName: Computer Science Database
  customDbUrl:
  eissn: 1076-9757
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0019428
  issn: 1076-9757
  databaseCode: K7-
  dateStart: 19930101
  isFulltext: true
  titleUrlDefault: http://search.proquest.com/compscijour
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: ProQuest Central (subscription)
  customDbUrl:
  eissn: 1076-9757
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0019428
  issn: 1076-9757
  databaseCode: BENPR
  dateStart: 19930101
  isFulltext: true
  titleUrlDefault: https://www.proquest.com/central
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: Publicly Available Content Database
  customDbUrl:
  eissn: 1076-9757
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0019428
  issn: 1076-9757
  databaseCode: PIMPY
  dateStart: 19930101
  isFulltext: true
  titleUrlDefault: http://search.proquest.com/publiccontent
  providerName: ProQuest
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lb9QwELaWwoFLeYvyqHyAU5US29nYPpYVCxJoW4kW9RbZiaOugLQKu1XL_-L_MWM7j0IP5cDFWnnX3ijzaR72zDeEvFLaIeVKljjFTJIpWyda2QoGwYwEA6OsLxT-JBcLdXysDyaTX10tzPk32TTq4kKf_VdRwxwIG0tn_0Hc_aYwAZ9B6DCC2GG8keA_w1v39VAHSCSB1MHrjo055Ae8R47qHWTlgG8XIQ3cszKgcoBAGcNwPEffB23yPZZpjn3YoZrM-7H4CJGGYjnm94w0Qv1x8wxkiNfyb2chO3u-br8O0PyCiScePOaktxR7l9Upert6Hm-r3OXJsAQ30_lPF0t29mG3nTDXdplG8TiDZ6PjjKCBU5knWgbW6l13zVxU26EBVNS7gsuRCRehpcxf1gFcF9-VYNnuMjAROjQNukrC_Ydx7FMWMViC9QWuLljhV98it7mcaswk_CiT_vJKZzxUYManjvUWsPrN-L9HntDIpTm8TzajDOlewNADMnHNQ3Kv6_NBo9p_RI46SNEAKTpAinpIUQ8pGiBFO0hRgBS9Aik6htRjcjR_dzj7kMR2HEkJceoqEU6mkhtdY2sQW0ohwGiDh5nmnJtqykVZKsuEZdXU5VmthU0dYzXECxCS64qJJ2SjOW3cU0IhjmZK2srmUmQZr4xxRqe1NKAatCn5FnndvZriLLCuFNcJ4NkNf_ec3B2w9oJsrNq1e0nulOer5Y9225_DbHsZ_gZK_nnC
linkProvider ProQuest
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Scalable+Primal+Heuristics+Using+Graph+Neural+Networks+for+Combinatorial+Optimization&rft.jtitle=The+Journal+of+artificial+intelligence+research&rft.au=Cant%C3%BCrk%2C+Furkan&rft.au=Varol%2C+Taha&rft.au=Aydo%C4%9Fan%2C+Reyhan&rft.au=%C3%96zener%2C+Okan+%C3%96rsan&rft.date=2024-01-01&rft.issn=1076-9757&rft.eissn=1076-9757&rft.volume=80&rft.spage=327&rft.epage=376&rft_id=info:doi/10.1613%2Fjair.1.14972&rft.externalDBID=n%2Fa&rft.externalDocID=10_1613_jair_1_14972
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1076-9757&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1076-9757&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1076-9757&client=summon