Deep Clustering With Variational Autoencoder

An autoencoder that learns a latent space in an unsupervised manner has many applications in signal processing. However, the latent space of an autoencoder does not pursue the same clustering goal as Kmeans or GMM. A recent work proposes to artificially re-align each point in the latent space of an...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE signal processing letters Jg. 27; S. 231 - 235
Hauptverfasser:	Lim, Kart-Leong, Jiang, Xudong, Yi, Chenyu
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	New York IEEE 2020 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Bayes methods Clustering Datasets Energy management Euclidean geometry Gaussian distribution Optimization Photovoltaic cells Probabilistic logic Probabilistic models Probability distribution Random variables Signal processing Statistical analysis
ISSN:	1070-9908, 1558-2361
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	An autoencoder that learns a latent space in an unsupervised manner has many applications in signal processing. However, the latent space of an autoencoder does not pursue the same clustering goal as Kmeans or GMM. A recent work proposes to artificially re-align each point in the latent space of an autoencoder to its nearest class neighbors during training (Song et al. 2013). The resulting new latent space is found to be much more suitable for clustering, since clustering information is used. Inspired by previous works (Song et al. 2013), in this letter we propose several extensions to this technique. First, we propose a probabilistic approach to generalize Song's approach, such that Euclidean distance in the latent space is now represented by KL divergence. Second, as a consequence of this generalization we can now use probability distributions as inputs rather than points in the latent space. Third, we propose using Bayesian Gaussian mixture model for clustering in the latent space. We demonstrated our proposed method on digit recognition datasets, MNIST, USPS and SHVN as well as scene datasets, Scene15 and MIT67 with interesting findings.
AbstractList	An autoencoder that learns a latent space in an unsupervised manner has many applications in signal processing. However, the latent space of an autoencoder does not pursue the same clustering goal as Kmeans or GMM. A recent work proposes to artificially re-align each point in the latent space of an autoencoder to its nearest class neighbors during training (Song et al. 2013). The resulting new latent space is found to be much more suitable for clustering, since clustering information is used. Inspired by previous works (Song et al. 2013), in this letter we propose several extensions to this technique. First, we propose a probabilistic approach to generalize Song's approach, such that Euclidean distance in the latent space is now represented by KL divergence. Second, as a consequence of this generalization we can now use probability distributions as inputs rather than points in the latent space. Third, we propose using Bayesian Gaussian mixture model for clustering in the latent space. We demonstrated our proposed method on digit recognition datasets, MNIST, USPS and SHVN as well as scene datasets, Scene15 and MIT67 with interesting findings.
Author	Yi, Chenyu Jiang, Xudong Lim, Kart-Leong
Author_xml	– sequence: 1 givenname: Kart-Leong orcidid: 0000-0001-9050-2300 surname: Lim fullname: Lim, Kart-Leong email: lkartl@yahoo.com.sg organization: Rapid-Rich Object Search Lab, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore – sequence: 2 givenname: Xudong orcidid: 0000-0002-9104-2315 surname: Jiang fullname: Jiang, Xudong email: exdjiang@ntu.edu.sg organization: Rapid-Rich Object Search Lab, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore – sequence: 3 givenname: Chenyu orcidid: 0000-0001-5002-6549 surname: Yi fullname: Yi, Chenyu email: yich0003@e.ntu.edu.sg organization: Rapid-Rich Object Search Lab, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore
BookMark	eNp9kL1PwzAQxS1UJNrCjsQSiZWU8zl27LEq5UOqBBJfo-UkDrgKSbGdgf-elFYMDEzvhve7e_cmZNR2rSXklMKMUlCXq8eHGQLCDJXgDOUBGVPOZYpM0NEwQw6pUiCPyCSENQBIKvmYXFxZu0kWTR-i9a59S15dfE9ejHcmuq41TTLvY2fbsqusPyaHtWmCPdnrlDxfL58Wt-nq_uZuMV-lJSoaU14JQGUlA8jqQTCnWDFOZcWyvDLMgCoKVco6kxYLDgpriVxQUxhRC1GwKTnf7d347rO3Iep11_shTNDIOCLNpMwGF-xcpe9C8LbWG-8-jP_SFPS2Ez10ored6H0nAyL-IKWLP49Gb1zzH3i2A5219veOVDwfgrNvgO1uQw
CODEN	ISPLEM
CitedBy_id	crossref_primary_10_1016_j_neucom_2021_08_128 crossref_primary_10_3390_app12063008 crossref_primary_10_1016_j_chemolab_2023_105029 crossref_primary_10_1016_j_compbiomed_2024_109632 crossref_primary_10_1109_TNNLS_2023_3307126 crossref_primary_10_1016_j_bspc_2023_105202 crossref_primary_10_1109_ACCESS_2020_3020844 crossref_primary_10_1016_j_engappai_2024_109562 crossref_primary_10_1016_j_sigpro_2024_109597 crossref_primary_10_1016_j_ins_2024_121371 crossref_primary_10_1038_s41598_024_76047_z crossref_primary_10_1080_19466315_2024_2368787 crossref_primary_10_1016_j_patcog_2022_108611 crossref_primary_10_1088_2632_2153_ad622f crossref_primary_10_1016_j_commatsci_2024_113056 crossref_primary_10_3390_pr12122760 crossref_primary_10_1016_j_patrec_2021_08_004 crossref_primary_10_1002_aps3_11559 crossref_primary_10_1016_j_imavis_2023_104746 crossref_primary_10_1109_TGRS_2024_3374597 crossref_primary_10_1016_j_physa_2025_130395 crossref_primary_10_1080_01431161_2022_2061317 crossref_primary_10_1007_s00371_024_03303_8 crossref_primary_10_1109_ACCESS_2021_3064854 crossref_primary_10_1109_TIM_2022_3227553 crossref_primary_10_1016_j_elerap_2023_101250 crossref_primary_10_1016_j_rineng_2025_105232 crossref_primary_10_1016_j_imavis_2024_105347 crossref_primary_10_1061_JAEEEZ_ASENG_4947 crossref_primary_10_1109_TFUZZ_2024_3462545 crossref_primary_10_1109_JSTARS_2020_3040218 crossref_primary_10_1016_j_rspp_2024_100154 crossref_primary_10_1109_TSM_2025_3579031 crossref_primary_10_1016_j_cie_2022_108695 crossref_primary_10_1109_ACCESS_2021_3115024 crossref_primary_10_1016_j_jnca_2020_102854 crossref_primary_10_1109_ACCESS_2024_3368795 crossref_primary_10_1016_j_geoen_2025_214170 crossref_primary_10_3390_e23111537 crossref_primary_10_1109_ACCESS_2022_3213674 crossref_primary_10_1109_ACCESS_2023_3238795 crossref_primary_10_3390_app14020719 crossref_primary_10_1016_j_knosys_2022_108827 crossref_primary_10_1631_FITEE_2200344 crossref_primary_10_3390_sym17020161 crossref_primary_10_1016_j_trc_2020_102834 crossref_primary_10_1007_s00138_022_01338_2 crossref_primary_10_1177_87552930221104838 crossref_primary_10_1109_TGRS_2023_3267070 crossref_primary_10_1007_s13042_022_01557_z crossref_primary_10_1007_s10994_024_06683_z crossref_primary_10_1016_j_asoc_2024_112107 crossref_primary_10_1109_RBME_2022_3185953 crossref_primary_10_1016_j_eswa_2025_128578 crossref_primary_10_1016_j_energy_2022_125969 crossref_primary_10_1016_j_neucom_2025_130901
Cites_doi	10.1109/LSP.2017.2697970 10.1016/j.patcog.2018.05.019 10.1109/LSP.2017.2672753 10.1109/ICASSP.2019.8683497 10.1109/TPAMI.2008.258 10.1109/ACCESS.2018.2848210 10.1109/TKDE.2005.198 10.1109/ICASSP.2019.8682823 10.1109/LSP.2018.2843295 10.1109/LSP.2017.2752459 10.1109/MSP.2010.939041 10.1109/TNNLS.2017.2747861 10.24963/ijcai.2017/273 10.1109/CVPR.2006.68 10.1109/5.726791 10.1109/IJCNN.2018.8489030 10.1007/978-3-319-71246-8_49 10.1109/TASLP.2019.2917232 10.1109/ICASSP.2019.8682546 10.1109/TASLP.2018.2797420 10.1109/CVPR.2009.5206537 10.1109/LSP.2019.2913022 10.1109/TPAMI.2019.2913863 10.1007/978-0-387-21606-5 10.1109/ACCESS.2019.2920592 10.1007/978-3-642-41822-8_15 10.1109/TSP.2019.2899294 10.1109/TIP.2018.2848470 10.1109/CVPR.2016.90 10.1109/TNNLS.2019.2900734 10.1109/LSP.2019.2929440
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/LSP.2020.2965328
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1558-2361
EndPage	235
ExternalDocumentID	10_1109_LSP_2020_2965328 8957256
Genre	orig-research
GroupedDBID	-~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 85S 97E AAJGR AARMG AASAJ AAWTH AAYJJ ABAZT ABFSI ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 E.L EBS EJD F5P HZ~ H~9 ICLAB IFIPE IFJZH IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS TAE TN5 VH1 AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c291t-5d6029e83004fe832712d3518d347da3a09bb9c8f48e2b5092f82561aba6f66b3
IEDL.DBID	RIE
ISICitedReferencesCount	74
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000516622400003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1070-9908
IngestDate	Sun Nov 09 08:25:46 EST 2025 Sat Nov 29 01:48:53 EST 2025 Tue Nov 18 19:48:08 EST 2025 Wed Aug 27 02:32:41 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c291t-5d6029e83004fe832712d3518d347da3a09bb9c8f48e2b5092f82561aba6f66b3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-9104-2315 0000-0001-5002-6549 0000-0001-9050-2300
PQID	2352214884
PQPubID	75747
PageCount	5
ParticipantIDs	proquest_journals_2352214884 crossref_primary_10_1109_LSP_2020_2965328 crossref_citationtrail_10_1109_LSP_2020_2965328 ieee_primary_8957256
PublicationCentury	2000
PublicationDate	20200000 2020-00-00 20200101
PublicationDateYYYYMMDD	2020-01-01
PublicationDate_xml	– year: 2020 text: 20200000
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE signal processing letters
PublicationTitleAbbrev	LSP
PublicationYear	2020
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref34 ref15 ref36 ref14 ref31 ref30 ref33 maddison (ref25) 2016 ref32 yang (ref37) 0; 70 ref10 lim (ref24) 2017 kingma (ref19) 0 ref2 ref1 ref39 ref17 ref38 ref16 ref18 bishop (ref5) 2006 ref23 ji (ref12) 0 ref20 ref22 jang (ref11) 2016 ref21 ref28 ref27 ref29 ref8 ref7 xie (ref35) 0 ref9 netzer (ref26) 2011 ref4 ref3 ref6 ref40
References_xml	– ident: ref32 doi: 10.1109/LSP.2017.2697970 – start-page: 478 year: 0 ident: ref35 article-title: Unsupervised deep embedding for clustering analysis publication-title: Proc Int Conf Mach Learn – ident: ref23 doi: 10.1016/j.patcog.2018.05.019 – ident: ref7 doi: 10.1109/LSP.2017.2672753 – ident: ref10 doi: 10.1109/ICASSP.2019.8683497 – ident: ref14 doi: 10.1109/TPAMI.2008.258 – ident: ref33 doi: 10.1109/ACCESS.2018.2848210 – ident: ref6 doi: 10.1109/TKDE.2005.198 – year: 2006 ident: ref5 publication-title: Pattern Recognition and Machine Learning – ident: ref27 doi: 10.1109/ICASSP.2019.8682823 – ident: ref39 doi: 10.1109/LSP.2018.2843295 – ident: ref2 doi: 10.1109/LSP.2017.2752459 – ident: ref15 doi: 10.1109/MSP.2010.939041 – year: 0 ident: ref19 article-title: Stochastic gradient VB and the variational auto-encoder publication-title: Proc Intl Conf on Learning Representations – ident: ref4 doi: 10.1109/TNNLS.2017.2747861 – ident: ref16 doi: 10.24963/ijcai.2017/273 – volume: 70 start-page: 3861 year: 0 ident: ref37 article-title: Towards k-means-friendly spaces: Simultaneous deep learning and clustering publication-title: Proc 34th Int Conf Mach Learn – ident: ref20 doi: 10.1109/CVPR.2006.68 – ident: ref21 doi: 10.1109/5.726791 – ident: ref30 doi: 10.1109/IJCNN.2018.8489030 – start-page: 24 year: 0 ident: ref12 article-title: Deep subspace clustering networks publication-title: Proc Advances Neural Inform Process Syst – ident: ref34 doi: 10.1007/978-3-319-71246-8_49 – year: 2016 ident: ref11 article-title: Categorical reparameterization with gumbel-softmax – ident: ref17 doi: 10.1109/TASLP.2019.2917232 – ident: ref22 doi: 10.1109/ICASSP.2019.8682546 – year: 2011 ident: ref26 article-title: Reading digits in natural images with unsupervised feature learning – ident: ref40 doi: 10.1109/TASLP.2018.2797420 – ident: ref29 doi: 10.1109/CVPR.2009.5206537 – year: 2016 ident: ref25 article-title: The concrete distribution: A continuous relaxation of discrete random variables – ident: ref1 doi: 10.1109/LSP.2019.2913022 – ident: ref38 doi: 10.1109/TPAMI.2019.2913863 – ident: ref8 doi: 10.1007/978-0-387-21606-5 – ident: ref13 doi: 10.1109/ACCESS.2019.2920592 – ident: ref31 doi: 10.1007/978-3-642-41822-8_15 – ident: ref3 doi: 10.1109/TSP.2019.2899294 – ident: ref28 doi: 10.1109/TIP.2018.2848470 – ident: ref9 doi: 10.1109/CVPR.2016.90 – start-page: 1 year: 2017 ident: ref24 article-title: Map approximation to the variational bayes Gaussian mixture model and application publication-title: Soft Comput – ident: ref36 doi: 10.1109/TNNLS.2019.2900734 – ident: ref18 doi: 10.1109/LSP.2019.2929440
SSID	ssj0008185
Score	2.5752845
Snippet	An autoencoder that learns a latent space in an unsupervised manner has many applications in signal processing. However, the latent space of an autoencoder...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	231
SubjectTerms	Bayes methods Clustering Datasets Energy management Euclidean geometry Gaussian distribution Optimization Photovoltaic cells Probabilistic logic Probabilistic models Probability distribution Random variables Signal processing Statistical analysis
Title	Deep Clustering With Variational Autoencoder
URI	https://ieeexplore.ieee.org/document/8957256 https://www.proquest.com/docview/2352214884
Volume	27
WOSCitedRecordID	wos000516622400003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2361 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0008185 issn: 1070-9908 databaseCode: RIE dateStart: 19940101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFH_M4UEPfk1xOqUHL8K6pWmb5B3HdHiQMfBrt9J8FAdjG1vn32_SdmOgCJ5aSlLKL8177_eS_B7AHaIiNkoOfR5J4kfaTimpTexLg4qFTsAKs6LYBB8OxXiMoxq0t2dhjDHF5jPTcbfFWr6eq7VLlXUFxty66D3Y45yXZ7W2Vtc5nnJ_IfGthRWbJUmC3eeXkSWClHQosjh0ddd3XFBRU-WHIS68y-D4f991AkdVFOn1ymE_hZqZncHhjrZgA9oPxiy8_nTtlBDsE-9jkn9675YaV-k_r7fO507GUpvlObwNHl_7T35VGsFXFIPcjzUjFI1welmZvVAeUB3GgdBhxHUapgSlRCWySBgqbVBAM0sFWZDKlGWMyfAC6rP5zFyClwrFWWyZFxMqCkiYUqYZxVQiZhHyrAndDVqJqnTDXfmKaVLwB4KJxTdx-CYVvk243_ZYlJoZf7RtODy37Soom9DaDEhSTapVQl2waOmbiK5-73UNB-7dZYakBfV8uTY3sK--8slqeVv8L9_k47ph
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFD7MKagP3sXp1D74IliXpmma8zimojjHwOtbaS7FgWyydf5-k7YbA0XwqaUktHxpzjnfSfIdgDNERWyUHPoxk8Rn2k4pqU3kS4OKh07ACrOi2ETc64m3N-zX4GJ-FsYYU2w-M5futljL1yM1damylsAoti56CZYjxmhQntaa213nesodhsS3NlbMFiUJtrqPfUsFKbmkyKPQVV5fcEJFVZUfprjwLzeb__uyLdio4kivXQ78NtTMcAfWF9QFd-HiyphPr_MxdVoI9on3OsjfvRdLjqsEoNee5iMnZKnNeA-eb66fOrd-VRzBVxSD3I80JxSNcIpZmb3QOKA6jAKhQxbrNEwJSolKZEwYKm1YQDNLBnmQypRnnMtwH-rD0dAcgJcKFfPIci8uFAtImFKuOcVUImYM46wBrRlaiaqUw10Bi4-kYBAEE4tv4vBNKnwbcD7v8VmqZvzRdtfhOW9XQdmA5mxAkmpaTRLqwkVL4AQ7_L3XKazePj10k-5d7_4I1tx7ynxJE-r5eGqOYUV95YPJ-KT4d74Bx3u9qA
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Deep+Clustering+With+Variational+Autoencoder&rft.jtitle=IEEE+signal+processing+letters&rft.au=Lim%2C+Kart-Leong&rft.au=Jiang%2C+Xudong&rft.au=Yi%2C+Chenyu&rft.date=2020&rft.pub=IEEE&rft.issn=1070-9908&rft.volume=27&rft.spage=231&rft.epage=235&rft_id=info:doi/10.1109%2FLSP.2020.2965328&rft.externalDocID=8957256
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1070-9908&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1070-9908&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1070-9908&client=summon