Convergence of a Fast Hierarchical Alternating Least Squares Algorithm for Nonnegative Matrix Factorization

The hierarchical alternating least squares (HALS) algorithms are powerful tools for nonnegative matrix factorization (NMF), among which the Fast-HALS, proposed in [A. Cichocki and A.-H. Phan, 2009], is one of the most efficient. This paper investigates the convergence of Fast-HALS. First, a more gen...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on knowledge and data engineering Vol. 36; no. 1; pp. 77 - 89
Main Authors: Hou, Liangshao, Chu, Delin, Liao, Li-Zhi
Format: Journal Article
Language:English
Published: New York IEEE 01.01.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Algorithm design and analysis
Algorithms
Approximation algorithms
Columns (structural)
Convergence
Factorization
Kurdyka-Łojasiewicz property
Least squares
Least squares approximations
nonnegative matrix factorization
the fast hierarchical alternating least squares algorithm
ISSN:1041-4347, 1558-2191
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract The hierarchical alternating least squares (HALS) algorithms are powerful tools for nonnegative matrix factorization (NMF), among which the Fast-HALS, proposed in [A. Cichocki and A.-H. Phan, 2009], is one of the most efficient. This paper investigates the convergence of Fast-HALS. First, a more general weak convergence (converged subsequences exist and converge to the stationary point set) is established without any assumption, while most existing results assume all the columns of iterates are strictly away from the origin. Then, a simplified strong convergence (the entire sequence converges to a stationary point) proof is provided. The existing strong convergence is attributed to the block prox-linear (BPL) method, which is a more general framework including Fast-HALS as a special case. So, the convergence proof under BPL is quite complex. Our simplified proof explores the structure of Fast-HALS and can be regarded as a complement to the results under BPL. In addition, some numerical verifications are presented.
AbstractList The hierarchical alternating least squares (HALS) algorithms are powerful tools for nonnegative matrix factorization (NMF), among which the Fast-HALS, proposed in [A. Cichocki and A.-H. Phan, 2009], is one of the most efficient. This paper investigates the convergence of Fast-HALS. First, a more general weak convergence (converged subsequences exist and converge to the stationary point set) is established without any assumption, while most existing results assume all the columns of iterates are strictly away from the origin. Then, a simplified strong convergence (the entire sequence converges to a stationary point) proof is provided. The existing strong convergence is attributed to the block prox-linear (BPL) method, which is a more general framework including Fast-HALS as a special case. So, the convergence proof under BPL is quite complex. Our simplified proof explores the structure of Fast-HALS and can be regarded as a complement to the results under BPL. In addition, some numerical verifications are presented.
Author Liao, Li-Zhi
Chu, Delin
Hou, Liangshao
Author_xml – sequence: 1
  givenname: Liangshao
  orcidid: 0000-0001-8404-8615
  surname: Hou
  fullname: Hou, Liangshao
  email: houliangshao@hkbu.edu.hk
  organization: Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong
– sequence: 2
  givenname: Delin
  orcidid: 0000-0001-7736-7490
  surname: Chu
  fullname: Chu, Delin
  email: matchudl@nus.edu.sg
  organization: Department of Mathematics, National University of Singapore, Singapore
– sequence: 3
  givenname: Li-Zhi
  orcidid: 0000-0002-0588-7953
  surname: Liao
  fullname: Liao, Li-Zhi
  email: liliao@hkbu.edu.hk
  organization: Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong
BookMark eNp9ULlOAzEQtRBInB-ARGGJeoOv3bXLKCSACFAA9crxzgbDxia2g4CvxyEUiIJqRvOO0Xv7aNt5BwgdUzKglKizh-vz8YARxgec1YpXagvt0bKUBaOKbuedCFoILupdtB_jMyFE1pLuoZeRd28Q5uAMYN9hjSc6JnxpIehgnqzRPR72CYLTybo5nsIavl-udICYkbkPNj0tcOcDvvXOwTzz3gDf6BTsezYzKTM-89G7Q7TT6T7C0c88QI-T8cPospjeXVyNhtPCMCVSYUDXJShJWiBVWUlT1qXqCBNUcgpcCDXjZdWJkhlSwaylbadnvNXSCOjavB6g043va_DLFcTUPPtVDtDHhkmlRCWzSWbRDcsEH2OArnkNdqHDR0NJs-60WXfarDttfjrNmvqPxtj0nS0Fbft_lScbpQWAX58oZzki_wKJqofv
CODEN ITKEEH
CitedBy_id crossref_primary_10_1016_j_neucom_2024_128440
crossref_primary_10_36930_40350120
Cites_doi 10.1155/2008/939567
10.1090/S0002-9947-09-05048-X
10.1007/s10915-017-0531-7
10.1137/07069239X
10.1016/j.csda.2006.11.006
10.1137/20m1352405
10.1007/s10107-007-0133-5
10.1109/tpami.2022.3206465
10.1007/978-3-540-74494-8_22
10.1137/050644641
10.1587/transfun.e92.a.708
10.1093/bioinformatics/btm134
10.1007/s10107-011-0484-9
10.1287/moor.1100.0449
10.1162/neco.2007.19.10.2756
10.1007/bf01584660
10.1137/120891009
10.1109/ICASSP.2009.4959890
10.1002/env.3170050203
10.5802/aif.1638
10.1016/S0167-6377(99)00074-7
10.1007/3-540-31247-1
10.1145/860435.860485
10.1007/11785231_91
10.1007/s10898-022-01167-7
10.1007/s10898-014-0247-2
10.1109/TKDE.2012.51
10.1007/s10915-017-0376-0
10.1057/palgrave.jors.2600425
10.1371/journal.pcbi.1000029
10.5802/aif.1384
10.1137/110821172
10.1137/1.9781611972771.31
10.1137/1.9781611976410
10.1137/120887795
10.1016/j.patrec.2009.12.023
10.1038/44565
10.1007/s10898-013-0035-4
10.1109/camsap.2015.7383726
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
DOI 10.1109/TKDE.2023.3279369
DatabaseName IEEE Xplore (IEEE)
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Technology Research Database
Computer and Information Systems Abstracts – Academic
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
DatabaseTitleList
Technology Research Database
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
Computer Science
EISSN 1558-2191
EndPage 89
ExternalDocumentID 10_1109_TKDE_2023_3279369
10132568
Genre orig-research
GrantInformation_xml – fundername: NUS
  grantid: A-0004252-00-00; A-8000430-00-00
– fundername: General Research Fund
  grantid: HKBU12302019; HKBU12300920
GroupedDBID -~X
.DC
0R~
1OL
29I
4.4
5GY
5VS
6IK
97E
9M8
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABFSI
ABQJQ
ABVLG
ACGFO
ACIWK
AENEX
AETIX
AGQYO
AGSQL
AHBIQ
AI.
AIBXA
AKJIK
AKQYR
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ASUFR
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
E.L
EBS
EJD
F5P
HZ~
H~9
ICLAB
IEDLZ
IFIPE
IFJZH
IPLJI
JAVBF
LAI
M43
MS~
O9-
OCL
P2P
PQQKQ
RIA
RIE
RNI
RNS
RXW
RZB
TAE
TAF
TN5
UHB
VH1
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
ID FETCH-LOGICAL-c294t-cea75e980de06568c5759f0241831e3449b356f452c06ebd1dfab3da8c4efdab3
IEDL.DBID RIE
ISICitedReferencesCount 2
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001124222100001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1041-4347
IngestDate Sun Nov 09 08:22:03 EST 2025
Tue Nov 18 21:29:00 EST 2025
Sat Nov 29 02:36:07 EST 2025
Wed Aug 27 02:05:28 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 1
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c294t-cea75e980de06568c5759f0241831e3449b356f452c06ebd1dfab3da8c4efdab3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0001-7736-7490
0000-0002-0588-7953
0000-0001-8404-8615
PQID 2899468344
PQPubID 85438
PageCount 13
ParticipantIDs crossref_primary_10_1109_TKDE_2023_3279369
proquest_journals_2899468344
crossref_citationtrail_10_1109_TKDE_2023_3279369
ieee_primary_10132568
PublicationCentury 2000
PublicationDate 2024-Jan.
2024-1-00
20240101
PublicationDateYYYYMMDD 2024-01-01
PublicationDate_xml – month: 01
  year: 2024
  text: 2024-Jan.
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE transactions on knowledge and data engineering
PublicationTitleAbbrev TKDE
PublicationYear 2024
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref35
ref34
ref15
ref37
ref14
ref36
ref30
ref11
ref33
ref10
ref32
ref2
ref1
ref17
ref39
ref16
ref38
ref19
ref18
Buciu (ref3) 2008; 3
ref24
Rockafellar (ref31) 2009
ref23
ref26
ref25
ref20
ref42
ref41
ref22
ref21
ref43
Hoyer (ref5) 2004; 5
ref28
Lee (ref12)
ref27
ref29
ref8
ref7
ref9
ref4
ref6
ref40
References_xml – ident: ref14
  doi: 10.1155/2008/939567
– volume-title: Variational Analysis
  year: 2009
  ident: ref31
– ident: ref37
  doi: 10.1090/S0002-9947-09-05048-X
– ident: ref2
  doi: 10.1007/s10915-017-0531-7
– ident: ref17
  doi: 10.1137/07069239X
– ident: ref19
  doi: 10.1016/j.csda.2006.11.006
– ident: ref20
  doi: 10.1137/20m1352405
– ident: ref41
  doi: 10.1007/s10107-007-0133-5
– ident: ref23
  doi: 10.1109/tpami.2022.3206465
– ident: ref21
  doi: 10.1007/978-3-540-74494-8_22
– ident: ref34
  doi: 10.1137/050644641
– ident: ref22
  doi: 10.1587/transfun.e92.a.708
– ident: ref8
  doi: 10.1093/bioinformatics/btm134
– ident: ref42
  doi: 10.1007/s10107-011-0484-9
– ident: ref33
  doi: 10.1287/moor.1100.0449
– ident: ref13
  doi: 10.1162/neco.2007.19.10.2756
– ident: ref39
  doi: 10.1007/bf01584660
– ident: ref38
  doi: 10.1137/120891009
– ident: ref10
  doi: 10.1109/ICASSP.2009.4959890
– ident: ref11
  doi: 10.1002/env.3170050203
– volume: 3
  start-page: 67
  issue: 3
  year: 2008
  ident: ref3
  article-title: Non-negative matrix factorization, a new tool for feature extraction: Theory and applications
  publication-title: Int. J. Comput. Commun. Control.
– ident: ref36
  doi: 10.5802/aif.1638
– ident: ref40
  doi: 10.1016/S0167-6377(99)00074-7
– ident: ref32
  doi: 10.1007/3-540-31247-1
– ident: ref43
  doi: 10.1145/860435.860485
– ident: ref15
  doi: 10.1007/11785231_91
– ident: ref28
  doi: 10.1007/s10898-022-01167-7
– ident: ref4
  doi: 10.1007/s10898-014-0247-2
– ident: ref7
  doi: 10.1109/TKDE.2012.51
– ident: ref26
  doi: 10.1007/s10915-017-0376-0
– ident: ref30
  doi: 10.1057/palgrave.jors.2600425
– ident: ref6
  doi: 10.1371/journal.pcbi.1000029
– ident: ref35
  doi: 10.5802/aif.1384
– ident: ref18
  doi: 10.1137/110821172
– ident: ref16
  doi: 10.1137/1.9781611972771.31
– ident: ref29
  doi: 10.1137/1.9781611976410
– ident: ref25
  doi: 10.1137/120887795
– ident: ref9
  doi: 10.1016/j.patrec.2009.12.023
– ident: ref1
  doi: 10.1038/44565
– ident: ref24
  doi: 10.1007/s10898-013-0035-4
– volume: 5
  start-page: 38
  issue: 9
  year: 2004
  ident: ref5
  article-title: Non-negative matrix factorization with sparseness constraints
  publication-title: J. Mach. Learn. Res.
– start-page: 556
  volume-title: Proc. Adv. Neural. Inf. Process. Syst.
  ident: ref12
  article-title: Algorithms for non-negative matrix factorization
– ident: ref27
  doi: 10.1109/camsap.2015.7383726
SSID ssj0008781
Score 2.4342341
Snippet The hierarchical alternating least squares (HALS) algorithms are powerful tools for nonnegative matrix factorization (NMF), among which the Fast-HALS, proposed...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 77
SubjectTerms Algorithm design and analysis
Algorithms
Approximation algorithms
Columns (structural)
Convergence
Factorization
Kurdyka-Łojasiewicz property
Least squares
Least squares approximations
nonnegative matrix factorization
the fast hierarchical alternating least squares algorithm
Title Convergence of a Fast Hierarchical Alternating Least Squares Algorithm for Nonnegative Matrix Factorization
URI https://ieeexplore.ieee.org/document/10132568
https://www.proquest.com/docview/2899468344
Volume 36
WOSCitedRecordID wos001124222100001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 1558-2191
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0008781
  issn: 1041-4347
  databaseCode: RIE
  dateStart: 19890101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFH644UEPTufE6ZQcPAmd_Zk0R5kbA3UITtittGk6h7PTrRP_fPPSbExEwVugSSh8ecl7efneB3BBY-46NGGW46XCUh4xJgnd2HKEDGyaJILapdgEGwzC0Yg_GLK65sJIKfXjM9nGps7lpzOxxKsyZeEqdgpoWIEKY7Qka6233ZBpRVIVXqigyPOZSWE6Nr8a3t5026gT3vZchgp23w4hraryYyvW50uv9s8_24c940iS6xL5A9iSeR1qK5EGYmy2DrsbFQcP4aWDr8w14VKSWUZi0osXBelPkIesZVHUlFNzSZiPyR1K-5DH9yXSlNSX8Ww-KZ5fiXJ1yQDfyIx14XByj5X-P9VkmAMw1M4GPPW6w07fMnoLlnC5X1hCxiyQPLRTqRwTGgoU78zUIa7M3pGe7_PEC2jmB66wqUxSJ83ixEvjUPgyS1XzCKr5LJfHQFweu4HwhM21i8ASoeLQ1PUSiSXMWNgEewVAJEwxctTEmEY6KLF5hJhFiFlkMGvC5XrIW1mJ46_ODQRpo2OJTxNaK5gjY6yLCGNOn6LgyMkvw05hR83ul1cvLagW86U8g23xUUwW83O9Dr8ABH_aWw
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3fS8MwED50CuqD0zlxOjUPPgmd_Zk2jzI3JvuB4IS9lTZN53BuunXin28uzcZEFHwLNEkLXy-5y-W7D-CKRsy2aOwblpNwQ3rEmCS0I8PiwjNpHHNq5mITfq8XDAbsQZPVFRdGCKEun4kaNlUuP5nyBR6VSQuXsZNHg03YQuksTddaLbyBrzRJZYAhwyLH9XUS0zLZTb9916ihUnjNsX3UsPu2DSldlR-LsdphmsV_ftsB7GtXktzm2B_ChpiUoLiUaSDaakuwt1Zz8Ahe6njPXFEuBZmmJCLNaJ6R1giZyEoYRU451seEkyHpoLgPeXxfIFFJPhlOZ6Ps-ZVIZ5f08JbMUJUOJ12s9f8pJ8MsgCZ3luGp2ejXW4ZWXDC4zdzM4CLyPcECMxHSNaEBR_nOVG7j0vAt4bguix2Ppq5nc5OKOLGSNIqdJAq4K9JENo-hMJlOxAkQm0W2xx1uMuUk-DGXkWhiO7HAImZ-UAFzCUDIdTlyVMUYhyosMVmImIWIWagxq8D1ashbXovjr85lBGmtY45PBapLmENtrvMQo06XouTI6S_DLmGn1e92ws59r30Gu_JNbn4QU4VCNluIc9jmH9loPrtQ_-QXi_HdpA
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Convergence+of+a+Fast+Hierarchical+Alternating+Least+Squares+Algorithm+for+Nonnegative+Matrix+Factorization&rft.jtitle=IEEE+transactions+on+knowledge+and+data+engineering&rft.au=Hou%2C+Liangshao&rft.au=Chu%2C+Delin&rft.au=Liao%2C+Li-Zhi&rft.date=2024-01-01&rft.issn=1041-4347&rft.eissn=1558-2191&rft.volume=36&rft.issue=1&rft.spage=77&rft.epage=89&rft_id=info:doi/10.1109%2FTKDE.2023.3279369&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TKDE_2023_3279369
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1041-4347&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1041-4347&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1041-4347&client=summon