A multi–modal unsupervised fault detection system based on power signals and thermal imaging via deep AutoEncoder neural network

In this paper a multi-modal unsupervised Deep Learning based algorithm for fault detection is proposed. Such method is applied to real data from a testing procedure implemented on an industrial production line. Both thermal images and current and power measurements coming from industrial refrigerato...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	Engineering applications of artificial intelligence Ročník 110; s. 104729
Hlavní autori:	Cordoni, Francesco, Bacchiega, Gianluca, Bondani, Giulio, Radu, Robert, Muradore, Riccardo
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	Elsevier Ltd 01.04.2022
Predmet:	Auto-encoder neural network Deep convolutional neural network Deep Learning Fault detection and isolation Neural networks Predictive maintenance Thermal camera Unsupervised learning Neural networks Fault detection and isolation Auto-encoder neural network Deep convolutional neural network Deep Learning Unsupervised learning Thermal camera Predictive maintenance
ISSN:	0952-1976, 1873-6769
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	In this paper a multi-modal unsupervised Deep Learning based algorithm for fault detection is proposed. Such method is applied to real data from a testing procedure implemented on an industrial production line. Both thermal images and current and power measurements coming from industrial refrigerators are collected. The considered dataset is highly unbalanced with the vast majority of samples being healthy. Thermal images are processed via a Deep Convolutional neural network. The features extracted from the thermal images are thus merged to structured data of power, current and temperature. Therefore, a Deep Auto-Encoder is trained on the dataset to signal anomalies corresponding to faults in the refrigerators. Three different methods are trained and compared: (1) an automatic method in which an expert extracts relevant features from thermal images without using the image recognition module; (2) a semi-automatic method where the convolutional neural network is applied to regions of interest within the thermal images selected by an expert operator; (3) a fully automatic method in which the Deep convolutional network processes the whole thermal image without any human intervention. The three methods show comparable results with nevertheless slight differences.
AbstractList	In this paper a multi-modal unsupervised Deep Learning based algorithm for fault detection is proposed. Such method is applied to real data from a testing procedure implemented on an industrial production line. Both thermal images and current and power measurements coming from industrial refrigerators are collected. The considered dataset is highly unbalanced with the vast majority of samples being healthy. Thermal images are processed via a Deep Convolutional neural network. The features extracted from the thermal images are thus merged to structured data of power, current and temperature. Therefore, a Deep Auto-Encoder is trained on the dataset to signal anomalies corresponding to faults in the refrigerators. Three different methods are trained and compared: (1) an automatic method in which an expert extracts relevant features from thermal images without using the image recognition module; (2) a semi-automatic method where the convolutional neural network is applied to regions of interest within the thermal images selected by an expert operator; (3) a fully automatic method in which the Deep convolutional network processes the whole thermal image without any human intervention. The three methods show comparable results with nevertheless slight differences.
ArticleNumber	104729
Author	Bacchiega, Gianluca Bondani, Giulio Muradore, Riccardo Cordoni, Francesco Radu, Robert
Author_xml	– sequence: 1 givenname: Francesco orcidid: 0000-0002-1295-7884 surname: Cordoni fullname: Cordoni, Francesco email: francesco.cordoni@unitn.it organization: University of Trento - Department of Civil, Environmental and Mechanical Engineering, Via Mesiano, 77, Trento, 38123, Italy – sequence: 2 givenname: Gianluca orcidid: 0000-0003-3923-4632 surname: Bacchiega fullname: Bacchiega, Gianluca email: bacchiega@irsweb.it organization: R&D, I.R.S. srl, Padova, Italy – sequence: 3 givenname: Giulio surname: Bondani fullname: Bondani, Giulio email: bondani@irsweb.it organization: R&D, I.R.S. srl, Padova, Italy – sequence: 4 givenname: Robert surname: Radu fullname: Radu, Robert email: radu@firstech.it organization: R&D, FirsT srl, Pordenone, Italy – sequence: 5 givenname: Riccardo orcidid: 0000-0002-0287-6896 surname: Muradore fullname: Muradore, Riccardo email: riccardo.muradore@univr.it organization: University of Verona - Department of Computer Science, Strada le Grazie, 15 Verona, 37134, Italy
BookMark	eNqFkM1OAyEURonRxFp9BcMLTAVmOgyJCxvjX2LiRteEgTuV2oEJMDXujK_gG_ok0lQ3blwRuJwv9ztHaN95BwidUjKjhNZnqxm4pRoGZWeMMJYfK87EHprQhpdFzWuxjyZEzFlBBa8P0VGMK0JI2VT1BH0scD-uk_16_-y9UWs8ujgOEDY2gsGdyjNsIIFO1jsc32KCHrdqO8z3wb9CwNEunVpHrJzB6RlCn2Nsr5bWLfHGqszDgBdj8ldOe5MBB2PIfxykVx9ejtFBl3E4-Tmn6On66vHytrh_uLm7XNwXuqQsFVC1dd6aQDWnZatbLoyeM6KVqgQrSSOapjN1KxpiOKcdFwK4Voa1Xdlwqkw5Ree7XB18jAE6qW1S214pKLuWlMitT7mSvz7l1qfc-cx4_QcfQm4Z3v4HL3Yg5HIbC0FGbcFpMDZkr9J4-1_ENwkFmnI
CitedBy_id	crossref_primary_10_1016_j_jtice_2023_105236 crossref_primary_10_1016_j_ssci_2024_106590 crossref_primary_10_1016_j_engappai_2023_106325 crossref_primary_10_3390_rs15010194 crossref_primary_10_1109_ACCESS_2025_3602914 crossref_primary_10_1016_j_measurement_2025_118325 crossref_primary_10_1016_j_engappai_2023_107389 crossref_primary_10_3390_s23115010 crossref_primary_10_1016_j_engappai_2024_109529 crossref_primary_10_1016_j_engappai_2023_105825 crossref_primary_10_1007_s10489_024_05332_1 crossref_primary_10_1016_j_engappai_2024_109374 crossref_primary_10_1016_j_engappai_2025_110332 crossref_primary_10_1088_1361_6501_acde97 crossref_primary_10_3390_en16124773 crossref_primary_10_3390_signals5010008 crossref_primary_10_1177_00368504251330003 crossref_primary_10_1016_j_csi_2024_103856 crossref_primary_10_1016_j_engappai_2025_111218
Cites_doi	10.3390/pr5030035 10.1016/j.engappai.2018.11.007 10.1016/j.promfg.2020.01.031 10.1016/j.cie.2019.106024 10.1016/j.ifacol.2020.12.2856 10.1561/2200000006 10.1016/j.promfg.2018.01.007
ContentType	Journal Article
Copyright	2022 Elsevier Ltd
Copyright_xml	– notice: 2022 Elsevier Ltd
DBID	AAYXX CITATION
DOI	10.1016/j.engappai.2022.104729
DatabaseName	CrossRef
DatabaseTitle	CrossRef
DatabaseTitleList
DeliveryMethod	fulltext_linktorsrc
Discipline	Applied Sciences Computer Science
EISSN	1873-6769
ExternalDocumentID	10_1016_j_engappai_2022_104729 S0952197622000392
GroupedDBID	--K --M .DC .~1 0R~ 1B1 1~. 1~5 29G 4.4 457 4G. 5GY 5VS 7-5 71M 8P~ 9JN AABNK AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXUO AAYFN ABBOA ABMAC ABXDB ABYKQ ACDAQ ACGFS ACNNM ACRLP ACZNC ADBBV ADEZE ADJOM ADMUD ADTZH AEBSH AECPX AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHJVU AHZHX AIALX AIEXJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ASPBG AVWKF AXJTR AZFZN BJAXD BKOJK BLXMC CS3 DU5 EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN G-2 G-Q GBLVA GBOLZ HLZ HVGLF HZ~ IHE J1W JJJVA KOM LG9 LY7 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 R2- RIG ROL RPZ SBC SDF SDG SDP SES SET SEW SPC SPCBC SST SSV SSZ T5K TN5 UHS WUQ ZMT ~G- 9DU AATTM AAXKI AAYWO AAYXX ABJNI ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP CITATION EFKBS ~HD
ID	FETCH-LOGICAL-c312t-e4b60030e4513bcb79dc520caa492308988fd6b980d771f799e7cad2bf3871ad3
ISICitedReferencesCount	22
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000795645300005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0952-1976
IngestDate	Sat Nov 29 07:07:19 EST 2025 Tue Nov 18 22:43:12 EST 2025 Fri Feb 23 02:39:21 EST 2024
IsPeerReviewed	true
IsScholarly	true
Keywords	Neural networks Fault detection and isolation Auto-encoder neural network Deep convolutional neural network Deep Learning Unsupervised learning Thermal camera Predictive maintenance
Language	English
LinkModel	OpenURL
MergedId	FETCHMERGED-LOGICAL-c312t-e4b60030e4513bcb79dc520caa492308988fd6b980d771f799e7cad2bf3871ad3
ORCID	0000-0003-3923-4632 0000-0002-0287-6896 0000-0002-1295-7884
ParticipantIDs	crossref_citationtrail_10_1016_j_engappai_2022_104729 crossref_primary_10_1016_j_engappai_2022_104729 elsevier_sciencedirect_doi_10_1016_j_engappai_2022_104729
PublicationCentury	2000
PublicationDate	April 2022 2022-04-00
PublicationDateYYYYMMDD	2022-04-01
PublicationDate_xml	– month: 04 year: 2022 text: April 2022
PublicationDecade	2020
PublicationTitle	Engineering applications of artificial intelligence
PublicationYear	2022
Publisher	Elsevier Ltd
Publisher_xml	– name: Elsevier Ltd
References	Bengio (b2) 2009; 2 Biasielli, Bolchini, Cassano, Miele (b3) 2019 Carvalho, Soares, Vita, Francisco, Basto, Alcalá (b5) 2019; 137 Vincent, Larochelle, Lajoie, Bengio, Manzagol (b13) 2010; 11 Tagawa, T., Tadokoro, Y., Yairi, T., Structured denoising autoencoder for fault detection and analysis. In: Asian Conference on Machine Learning, 2015, 96–111. Carletti, Masiero, Beghi, Susto (b4) 2019; 38 Maggipinto, Beghi, Susto (b8) 2019; 1 Zhang, Wang, Gao, Yan (b14) 2018; 19 Reis, Gins (b10) 2017; 5 Abadi, Agarwal, Barham, Brevdo, Chen, Citro, Corrado, Davis, Dean, Devin, Ghemawat, Goodfellow, Harp, Irving, Isard, Jia, Jozefowicz, Kaiser, Kudlur, Levenberg, Mané, Monga, Moore, Murray, Olah, Schuster, Shlens, Steiner, Sutskever, Talwar, Tucker, Vanhoucke, Vasudevan, Viégas, Vinyals, Warden, Wattenberg, Wicke, Yu, Zheng (b1) 2015 Chollet (b6) 2015 Muhuri, Shukla, Abraham (b9) 2019; 78 Zhao, Mukhopadhyay (b15) 2018 Cordoni, F.G., Bacchiega, G., Bondani, G., Radu, R., Muradore, R., A deep learning unsupervised approach for fault diagnosis of household appliances. In: IFAC Online, 21st IFAC World Conference, 2020. Simonyan, Zisserman (b11) 2014 Biasielli (10.1016/j.engappai.2022.104729_b3) 2019 10.1016/j.engappai.2022.104729_b7 Muhuri (10.1016/j.engappai.2022.104729_b9) 2019; 78 Reis (10.1016/j.engappai.2022.104729_b10) 2017; 5 Vincent (10.1016/j.engappai.2022.104729_b13) 2010; 11 Simonyan (10.1016/j.engappai.2022.104729_b11) 2014 Carvalho (10.1016/j.engappai.2022.104729_b5) 2019; 137 Zhang (10.1016/j.engappai.2022.104729_b14) 2018; 19 Carletti (10.1016/j.engappai.2022.104729_b4) 2019; 38 Abadi (10.1016/j.engappai.2022.104729_b1) 2015 Chollet (10.1016/j.engappai.2022.104729_b6) 2015 Zhao (10.1016/j.engappai.2022.104729_b15) 2018 10.1016/j.engappai.2022.104729_b12 Maggipinto (10.1016/j.engappai.2022.104729_b8) 2019; 1 Bengio (10.1016/j.engappai.2022.104729_b2) 2009; 2
References_xml	– volume: 2 start-page: 1 year: 2009 end-page: 127 ident: b2 article-title: Learning deep architectures for AI publication-title: Found. Trends Mach. Learn. – start-page: 704 year: 2019 end-page: 709 ident: b3 article-title: A smart fault detection scheme for reliable image processing applications publication-title: 2019 Design, Automation & Test In Europe Conference & Exhibition (DATE) – year: 2014 ident: b11 article-title: Very deep convolutional networks for large-scale image recognition – year: 2015 ident: b6 article-title: Keras. gitHub – volume: 137 year: 2019 ident: b5 article-title: A systematic literature review of machine learning methods applied to predictive maintenance publication-title: Comput. Ind. Eng. – reference: Cordoni, F.G., Bacchiega, G., Bondani, G., Radu, R., Muradore, R., A deep learning unsupervised approach for fault diagnosis of household appliances. In: IFAC Online, 21st IFAC World Conference, 2020. – volume: 78 start-page: 218 year: 2019 end-page: 235 ident: b9 article-title: Industry 4.0: A bibliometric analysis and detailed overview publication-title: Eng. Appl. Artif. Intell. – reference: Tagawa, T., Tadokoro, Y., Yairi, T., Structured denoising autoencoder for fault detection and analysis. In: Asian Conference on Machine Learning, 2015, 96–111. – year: 2015 ident: b1 article-title: TensorFlow: LArge-scale machine learning on heterogeneous systems – volume: 38 start-page: 233 year: 2019 end-page: 240 ident: b4 article-title: A deep learning approach for anomaly detection with industrial time series data: a refrigerators manufacturing case study publication-title: Procedia Manuf. – volume: 19 start-page: 42 year: 2018 end-page: 49 ident: b14 article-title: An image processing approach to machine fault diagnosis based on visual words representation publication-title: Procedia Manuf. – volume: 1 start-page: 187 year: 2019 end-page: 192 ident: b8 article-title: A deep learning-based approach to anomaly detection with 2-dimensional data in manufacturing publication-title: 2019 IEEE 17th International Conference on Industrial Informatics (INDIN) – volume: 11 start-page: 3371 year: 2010 end-page: 3408 ident: b13 article-title: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion publication-title: J. Mach. Learn. Res. – volume: 5 start-page: 35 year: 2017 ident: b10 article-title: Industrial process monitoring in the big data/industry 4.0 era: From detection, to diagnosis, to prognosis publication-title: Processes – start-page: 1966 year: 2018 end-page: 1970 ident: b15 article-title: A fault detection workflow using deep learning and image processing publication-title: SEG Technical Program Expanded Abstracts 2018 – year: 2014 ident: 10.1016/j.engappai.2022.104729_b11 – volume: 1 start-page: 187 year: 2019 ident: 10.1016/j.engappai.2022.104729_b8 article-title: A deep learning-based approach to anomaly detection with 2-dimensional data in manufacturing – year: 2015 ident: 10.1016/j.engappai.2022.104729_b1 – ident: 10.1016/j.engappai.2022.104729_b12 – volume: 5 start-page: 35 issue: 3 year: 2017 ident: 10.1016/j.engappai.2022.104729_b10 article-title: Industrial process monitoring in the big data/industry 4.0 era: From detection, to diagnosis, to prognosis publication-title: Processes doi: 10.3390/pr5030035 – start-page: 1966 year: 2018 ident: 10.1016/j.engappai.2022.104729_b15 article-title: A fault detection workflow using deep learning and image processing – volume: 78 start-page: 218 year: 2019 ident: 10.1016/j.engappai.2022.104729_b9 article-title: Industry 4.0: A bibliometric analysis and detailed overview publication-title: Eng. Appl. Artif. Intell. doi: 10.1016/j.engappai.2018.11.007 – volume: 38 start-page: 233 year: 2019 ident: 10.1016/j.engappai.2022.104729_b4 article-title: A deep learning approach for anomaly detection with industrial time series data: a refrigerators manufacturing case study publication-title: Procedia Manuf. doi: 10.1016/j.promfg.2020.01.031 – volume: 137 year: 2019 ident: 10.1016/j.engappai.2022.104729_b5 article-title: A systematic literature review of machine learning methods applied to predictive maintenance publication-title: Comput. Ind. Eng. doi: 10.1016/j.cie.2019.106024 – ident: 10.1016/j.engappai.2022.104729_b7 doi: 10.1016/j.ifacol.2020.12.2856 – volume: 2 start-page: 1 issue: 1 year: 2009 ident: 10.1016/j.engappai.2022.104729_b2 article-title: Learning deep architectures for AI publication-title: Found. Trends Mach. Learn. doi: 10.1561/2200000006 – volume: 19 start-page: 42 year: 2018 ident: 10.1016/j.engappai.2022.104729_b14 article-title: An image processing approach to machine fault diagnosis based on visual words representation publication-title: Procedia Manuf. doi: 10.1016/j.promfg.2018.01.007 – start-page: 704 year: 2019 ident: 10.1016/j.engappai.2022.104729_b3 article-title: A smart fault detection scheme for reliable image processing applications – year: 2015 ident: 10.1016/j.engappai.2022.104729_b6 – volume: 11 start-page: 3371 issue: Dec year: 2010 ident: 10.1016/j.engappai.2022.104729_b13 article-title: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion publication-title: J. Mach. Learn. Res.
SSID	ssj0003846
Score	2.4406645
Snippet	In this paper a multi-modal unsupervised Deep Learning based algorithm for fault detection is proposed. Such method is applied to real data from a testing...
SourceID	crossref elsevier
SourceType	Enrichment Source Index Database Publisher
StartPage	104729
SubjectTerms	Auto-encoder neural network Deep convolutional neural network Deep Learning Fault detection and isolation Neural networks Predictive maintenance Thermal camera Unsupervised learning
Title	A multi–modal unsupervised fault detection system based on power signals and thermal imaging via deep AutoEncoder neural network
URI	https://dx.doi.org/10.1016/j.engappai.2022.104729
Volume	110
WOSCitedRecordID	wos000795645300005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1873-6769 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0003846 issn: 0952-1976 databaseCode: AIEXJ dateStart: 19950201 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lj9MwELbKLgcuvNEuL_nALUppXrV9jFB5Ca0QWlBvkWM7kFU3iZqk2iPiL_DT-Af8Esax81hYseyBS9Q6GafpfPGMx-P5EHrGJRFwInTJggg3FCTQJW89l0QZvF5KCb8rpfTpHTk6ous1ez-b_ej3wuw2pCjo2Rmr_quqoQ2UrbfOXkHdQ6fQAJ9B6XAEtcPxnxQfmyTBPoshOC0lqKEt6rbS40INHmbG4QpHqkYZonBTztnRFk3q1YNKU6c5OrVDF1e2OZYwgm-c_NSwGu1yDvKqcuK2KVeF3hi_dXRtTLimMJnl52L-Y9VDZ7pk3mUhbLt0pY48ZFIfdFgcgdlx2fFOGRIQVYtyDL8K8SVXJjb8CoCuKdyGk2UhDWEVnGo3-SD1gct2TCqfhj1gxjxmy_TxS9_1mCGPGYZymyJrBmNdhcKEU_6wEyZkcTJXxWd4ap7P9S3mo8D5wty_GcwhjbHPkDtJ-n4S3U9i-rmG9n0SMbAW-_Gb1frt4CAE1Owf659gsnH94l90sc808YOOb6ObdgKDYwO8O2imirvolp3MYGsqamjq-UL6tnvoW4w7aP78-r0DJZ6CEnegxAMosQEl7kCJ4XsHSmxBiQGU2IISW1BiACXWoMQTUGIDSmxBeR99fLk6fvHatRwgrgg8v3FVmC61IVJh5AWpSAmTIvIXgnNdWXBBGaWZXKaMLiQhXkYYU0Rw6adZQInHZfAA7RVloQ4QJtmSEs6ZJxgHN5xyFjICHiyPVBr4Hj9EUf8vJ8IWyNc8LZvk73o-RM8HucqUiLlUgvVKTKyjaxzYBPB5iezDK9_tEboxvkCP0V6zbdUTdF3smrzePrXg_AW5sNXD
linkProvider	Elsevier
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+multi%E2%80%93modal+unsupervised+fault+detection+system+based+on+power+signals+and+thermal+imaging+via+deep+AutoEncoder+neural+network&rft.jtitle=Engineering+applications+of+artificial+intelligence&rft.au=Cordoni%2C+Francesco&rft.au=Bacchiega%2C+Gianluca&rft.au=Bondani%2C+Giulio&rft.au=Radu%2C+Robert&rft.date=2022-04-01&rft.issn=0952-1976&rft.volume=110&rft.spage=104729&rft_id=info:doi/10.1016%2Fj.engappai.2022.104729&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_engappai_2022_104729
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0952-1976&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0952-1976&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0952-1976&client=summon