The Computational Complexity of ReLU Network Training Parameterized by Data Dimensionality

Understanding the computational complexity of training simple neural networks with rectified linear units (ReLUs) has recently been a subject of intensive research. Closing gaps and complementing results from the literature, we present several results on the parameterized complexity of training two-...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:The Journal of artificial intelligence research Jg. 74; S. 1775 - 1790
Hauptverfasser: Froese, Vincent, Hertrich, Christoph, Niedermeier, Rolf
Format: Journal Article
Sprache:Englisch
Veröffentlicht: San Francisco AI Access Foundation 01.01.2022
Schlagworte:
Algorithms
Artificial intelligence
Complexity
Lower bounds
Neural networks
Parameterization
Parameters
Polynomials
Training
ISSN:1076-9757, 1076-9757, 1943-5037
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Abstract Understanding the computational complexity of training simple neural networks with rectified linear units (ReLUs) has recently been a subject of intensive research. Closing gaps and complementing results from the literature, we present several results on the parameterized complexity of training two-layer ReLU networks with respect to various loss functions. After a brief discussion of other parameters, we focus on analyzing the influence of the dimension d of the training data on the computational complexity. We provide running time lower bounds in terms of W[1]-hardness for parameter d and prove that known brute-force strategies are essentially optimal (assuming the Exponential Time Hypothesis). In comparison with previous work, our results hold for a broad(er) range of loss functions, including lp-loss for all p ∈ [0, ∞]. In particular, we improve a known polynomial-time algorithm for constant d and convex loss functions to a more general class of loss functions, matching our running time lower bounds also in these cases.
AbstractList Understanding the computational complexity of training simple neural networks with rectified linear units (ReLUs) has recently been a subject of intensive research. Closing gaps and complementing results from the literature, we present several results on the parameterized complexity of training two-layer ReLU networks with respect to various loss functions. After a brief discussion of other parameters, we focus on analyzing the influence of the dimension d of the training data on the computational complexity. We provide running time lower bounds in terms of W[1]-hardness for parameter d and prove that known brute-force strategies are essentially optimal (assuming the Exponential Time Hypothesis). In comparison with previous work, our results hold for a broad(er) range of loss functions, including lp-loss for all p ∈ [0, ∞]. In particular, we improve a known polynomial-time algorithm for constant d and convex loss functions to a more general class of loss functions, matching our running time lower bounds also in these cases.
Author Niedermeier, Rolf
Hertrich, Christoph
Froese, Vincent
Author_xml – sequence: 1
  givenname: Vincent
  surname: Froese
  fullname: Froese, Vincent
– sequence: 2
  givenname: Christoph
  surname: Hertrich
  fullname: Hertrich, Christoph
– sequence: 3
  givenname: Rolf
  surname: Niedermeier
  fullname: Niedermeier, Rolf
BookMark eNptkM1OwzAQhC1UJNrCjQewxJUWr5PYyRG1_EkVINReuFhOsgGXJC62KyhPT9pyQIjT7krfjHZmQHqtbZGQU2BjEBBdLLVxYxhDlMTygPSBSTHKZCJ7v_YjMvB-yRhkMU_75Hn-inRim9U66GBsq-vdVeOnCRtqK_qEswW9x_Bh3RudO21a077QR-10gwGd-cKS5hs61UHTqWmw9TuXTn1MDitdezz5mUOyuL6aT25Hs4ebu8nlbFREDMKo0BxkUqSYaYzzuMSiBIY8rbQuI85klos8jpKClVUiRCmzKqlKECzL85xzjKIhOdv7rpx9X6MPamnXrvvBKy5ZKgRAnHQU31OFs947rFRh9pFDF6pWwNS2Q7XtUIHaddiJzv-IVs402m3-x78BLvN3RA
CitedBy_id crossref_primary_10_1080_21681015_2023_2212006
crossref_primary_10_1016_j_cam_2025_116933
crossref_primary_10_1007_s10107_023_02016_5
crossref_primary_10_1016_j_disopt_2023_100795
crossref_primary_10_1137_22M1489332
crossref_primary_10_1007_s10107_024_02096_x
crossref_primary_10_1371_journal_pone_0322202
crossref_primary_10_1287_ijoc_2021_0225
ContentType Journal Article
Copyright 2022. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the associated terms available at https://www.jair.org/index.php/jair/about
Copyright_xml – notice: 2022. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the associated terms available at https://www.jair.org/index.php/jair/about
DBID AAYXX
CITATION
8FE
8FG
ABUWG
AFKRA
ARAPS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
GNUQQ
HCIFZ
JQ2
K7-
P62
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
DOI 10.1613/jair.1.13547
DatabaseName CrossRef
ProQuest SciTech Collection
ProQuest Technology Collection
ProQuest Central (Alumni)
ProQuest Central UK/Ireland
Advanced Technologies & Computer Science Collection
ProQuest Central Essentials
ProQuest Central
Technology collection
ProQuest One Community College
ProQuest Central
ProQuest Central Student
SciTech Premium Collection
ProQuest Computer Science Collection
Computer Science Database
ProQuest Advanced Technologies & Aerospace Collection
ProQuest Central Premium
ProQuest One Academic
Publicly Available Content Database
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
DatabaseTitle CrossRef
Publicly Available Content Database
Advanced Technologies & Aerospace Collection
Computer Science Database
ProQuest Central Student
Technology Collection
ProQuest One Academic Middle East (New)
ProQuest Advanced Technologies & Aerospace Collection
ProQuest Central Essentials
ProQuest Computer Science Collection
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest One Academic UKI Edition
ProQuest Central Korea
ProQuest Central (New)
ProQuest One Academic
ProQuest One Academic (New)
DatabaseTitleList CrossRef
Publicly Available Content Database
Database_xml – sequence: 1
  dbid: PIMPY
  name: Publicly Available Content Database
  url: http://search.proquest.com/publiccontent
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISSN 1076-9757
1943-5037
EndPage 1790
ExternalDocumentID 10_1613_jair_1_13547
GroupedDBID .DC
29J
2WC
5GY
5VS
AAKMM
AAKPC
AALFJ
AAYFX
AAYXX
ACGFO
ACM
ADBBV
ADBSK
ADMLS
AEFXT
AEJOY
AENEX
AFFHD
AFKRA
AFWXC
AKRVB
ALMA_UNASSIGNED_HOLDINGS
AMVHM
ARAPS
BCNDV
BENPR
BGLVJ
CCPQU
CITATION
E3Z
EBS
EJD
F5P
FRJ
FRP
GROUPED_DOAJ
GUFHI
HCIFZ
K7-
KQ8
LHSKQ
LPJ
OK1
OVT
P2P
PHGZM
PHGZT
PIMPY
PQGLB
RNS
TR2
XSB
8FE
8FG
ABUWG
AZQEC
DWQXO
GNUQQ
JQ2
P62
PKEHL
PQEST
PQQKQ
PQUKI
PRINS
PUEGO
ID FETCH-LOGICAL-c301t-ca2175c8e9ae4b4decd10e28faad32079b6b435c0df566d79f5fd1609bbb22e33
IEDL.DBID K7-
ISICitedReferencesCount 12
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000844951300001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1076-9757
IngestDate Sat Sep 06 14:47:57 EDT 2025
Sat Nov 29 05:27:06 EST 2025
Tue Nov 18 21:33:28 EST 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c301t-ca2175c8e9ae4b4decd10e28faad32079b6b435c0df566d79f5fd1609bbb22e33
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
OpenAccessLink https://www.proquest.com/docview/2708661145?pq-origsite=%requestingapplication%
PQID 2708661145
PQPubID 5160723
PageCount 16
ParticipantIDs proquest_journals_2708661145
crossref_citationtrail_10_1613_jair_1_13547
crossref_primary_10_1613_jair_1_13547
PublicationCentury 2000
PublicationDate 2022-01-01
PublicationDateYYYYMMDD 2022-01-01
PublicationDate_xml – month: 01
  year: 2022
  text: 2022-01-01
  day: 01
PublicationDecade 2020
PublicationPlace San Francisco
PublicationPlace_xml – name: San Francisco
PublicationTitle The Journal of artificial intelligence research
PublicationYear 2022
Publisher AI Access Foundation
Publisher_xml – name: AI Access Foundation
SSID ssj0019428
Score 2.4930341
Snippet Understanding the computational complexity of training simple neural networks with rectified linear units (ReLUs) has recently been a subject of intensive...
SourceID proquest
crossref
SourceType Aggregation Database
Enrichment Source
Index Database
StartPage 1775
SubjectTerms Algorithms
Artificial intelligence
Complexity
Lower bounds
Neural networks
Parameterization
Parameters
Polynomials
Training
Title The Computational Complexity of ReLU Network Training Parameterized by Data Dimensionality
URI https://www.proquest.com/docview/2708661145
Volume 74
WOSCitedRecordID wos000844951300001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVAON
  databaseName: DOAJ Directory of Open Access Journals
  customDbUrl:
  eissn: 1076-9757
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0019428
  issn: 1076-9757
  databaseCode: DOA
  dateStart: 19930101
  isFulltext: true
  titleUrlDefault: https://www.doaj.org/
  providerName: Directory of Open Access Journals
– providerCode: PRVPQU
  databaseName: ProQuest Central
  customDbUrl:
  eissn: 1076-9757
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0019428
  issn: 1076-9757
  databaseCode: BENPR
  dateStart: 19930101
  isFulltext: true
  titleUrlDefault: https://www.proquest.com/central
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: ProQuest Computer Science Database (NC LIVE)
  customDbUrl:
  eissn: 1076-9757
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0019428
  issn: 1076-9757
  databaseCode: K7-
  dateStart: 19930101
  isFulltext: true
  titleUrlDefault: http://search.proquest.com/compscijour
  providerName: ProQuest
– providerCode: PRVPQU
  databaseName: Publicly Available Content Database
  customDbUrl:
  eissn: 1076-9757
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0019428
  issn: 1076-9757
  databaseCode: PIMPY
  dateStart: 19930101
  isFulltext: true
  titleUrlDefault: http://search.proquest.com/publiccontent
  providerName: ProQuest
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LSwMxEA5qPXixPrFaSw56kmiTzTbNSXxUFLUsRUG9LHktVKTVdhXqr3eym_o46MXjZhNYdiYz30wm8yG0YzIqtYSwhCsWEW4kJVLbjBgZSwoOwx91FmQTottt393JJCTcxqGscmoTC0Nth8bnyA-YAPDdAvQeHz6_EM8a5U9XA4XGLKpQxqjX80tBPk8RJGflVTjRIlLEIhS-gwc7eFT90T71rA-eWOW7S_ppkQs3c1b97wcuocUAMPFRqRHLaMYNVlB1St6Aw15eRQ-gILgcDvnA4sn3x8wneJjhnru6xd2yShzfBCYJnChfzeUbPL87i_UEn6pc4VPPEVD294DVa-j2rHNzck4C0QIxsL9zYhQEJrFpO6kc19w6Y2nTsXamlI1YU0jd0gCrTNNmgP6skFmcWdpqSq01Yy6K1tHcYDhwGwiD0fIIEYCMdVzwuG1AS7iJeBZrAeCshvam_zo1oQu5J8N4Sn00ApJJvWRSmhaSqaHdz9nPZfeNX-bVpzJJwx4cp18C2fz79RZaYP5SQ5FYqaO5fPTqttG8ecv741EDVY473aTXKKL1RqFgMJZcXCf3Hxqk2b4
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1NT9tAEB2hUAkuQEsRX6V7KCfkEm_WWe-hQhUBERGiCAUJenH3yxJVlUBiQOFH9TcyY69pe6A3Dj3aXh_sfTvzdmZ2HsAnm8fKKNyWCM1bkbAqjpRxeWRVomJ0GJTqLMUmZL-fXl6qwRz8qs_CUFllbRNLQ-3GlmLk-1wi-W4je08Obm4jUo2i7GotoVHB4tTPHnDLNv3S7eD87nJ-fDQ8PImCqkBkEcxFZDWy8MSmXmkvjHDeurjpeZpr7Vq8KZVpG-QQtulypDpOqjzJXdxuKmMM554CoGjy5wWCPW3A_KB7Nrh6zlsowavDd7IdKZnIUGqPPnP_h76efI5JZ4KkXP50gn_7gNKxHS__b79kBZYChWZfK8y_hTk_egfLtTwFC9ZqFb7hEmDV7RDxLK-oA2gxY-OcnfveBetXdfBsGLQy2EBTvRq1sH70jpkZ6-hCsw6pIFQdTPDt93DxKl-4Bo3ReOTXgaFZJg6MVM15IUWSWlwHwrZEnhiJ9HMD9uq5zWzos05yHz8z2m8hEjJCQhZnJRI2YPd59E3VX-SFcds1BrJgZabZbwBs_vvxR1g4GZ71sl63f7oFi5yOcJRhpG1oFJM7_wHe2PviejrZCYBm8P21AfMEwLA2lA
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+Computational+Complexity+of+ReLU+Network+Training+Parameterized+by+Data+Dimensionality&rft.jtitle=The+Journal+of+artificial+intelligence+research&rft.au=Froese%2C+Vincent&rft.au=Hertrich%2C+Christoph&rft.au=Niedermeier%2C+Rolf&rft.date=2022-01-01&rft.issn=1076-9757&rft.eissn=1076-9757&rft.volume=74&rft.spage=1775&rft.epage=1790&rft_id=info:doi/10.1613%2Fjair.1.13547&rft.externalDBID=n%2Fa&rft.externalDocID=10_1613_jair_1_13547
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1076-9757&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1076-9757&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1076-9757&client=summon