Hierarchical Two-Stream Growing Self-Organizing Maps With Transience for Human Activity Recognition

The rapid growth in autonomous industrial environments has increased the need for intelligent video surveillance. As a predominant element of video surveillance, recognition of complex human movements is important in a wide range of surveillance applications. However, the current state-of-the-art vi...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE transactions on industrial informatics Ročník 16; číslo 12; s. 7756 - 7764
Hlavní autoři:	Nawaratne, Rashmika, Alahakoon, Damminda, De Silva, Daswin, Kumara, Harsha, Yu, Xinghuo
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	Piscataway IEEE 01.12.2020 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:	Activity recognition Algorithms Architecture Decision making Deep learning Feature extraction Forgetting hierarchical learning Histograms Human activity recognition human activity recognition (HAR) Machine learning Moving object recognition neural networks Neurons Optical flow Self organizing maps Self-organizing feature maps Surveillance Video data Video surveillance
ISSN:	1551-3203, 1941-0050
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Abstract	The rapid growth in autonomous industrial environments has increased the need for intelligent video surveillance. As a predominant element of video surveillance, recognition of complex human movements is important in a wide range of surveillance applications. However, the current state-of-the-art video surveillance techniques use supervised deep learning pipelines for human activity recognition (HAR). A key shortcoming of such techniques is the inability to learn from unlabeled video streams. To operate effectively in natural environments, video surveillance techniques have to be able to handle huge volumes of unlabeled video data, monitor and generate alerts and insights derived from multiple characteristics such as spatial structure, motion flow, color distribution, etc. Furthermore, most conventional learning systems lack memory persistence capability which can reduce the influence of outdated information in memory-guided decision-making resulting in limiting plasticity and overfitting based on specific past events. In this article, we propose a new adaptation of the Growing Self-Organizing Map (GSOM) to address these shortcomings by 1) adopting two proven concepts of traditional deep learning, hierarchical, and multistream learning, applied into GSOM self-structuring architecture to accommodate learning from unlabeled video data and their diverse characteristics, 2) address overfitting and the influence of outdated information on neural architecture by implementing a transience property in the algorithm. We demonstrate the proposed model using three benchmark video datasets and the results confirm its validity and usability for HAR.
AbstractList	The rapid growth in autonomous industrial environments has increased the need for intelligent video surveillance. As a predominant element of video surveillance, recognition of complex human movements is important in a wide range of surveillance applications. However, the current state-of-the-art video surveillance techniques use supervised deep learning pipelines for human activity recognition (HAR). A key shortcoming of such techniques is the inability to learn from unlabeled video streams. To operate effectively in natural environments, video surveillance techniques have to be able to handle huge volumes of unlabeled video data, monitor and generate alerts and insights derived from multiple characteristics such as spatial structure, motion flow, color distribution, etc. Furthermore, most conventional learning systems lack memory persistence capability which can reduce the influence of outdated information in memory-guided decision-making resulting in limiting plasticity and overfitting based on specific past events. In this article, we propose a new adaptation of the Growing Self-Organizing Map (GSOM) to address these shortcomings by 1) adopting two proven concepts of traditional deep learning, hierarchical, and multistream learning, applied into GSOM self-structuring architecture to accommodate learning from unlabeled video data and their diverse characteristics, 2) address overfitting and the influence of outdated information on neural architecture by implementing a transience property in the algorithm. We demonstrate the proposed model using three benchmark video datasets and the results confirm its validity and usability for HAR.
Author	De Silva, Daswin Kumara, Harsha Yu, Xinghuo Nawaratne, Rashmika Alahakoon, Damminda
Author_xml	– sequence: 1 givenname: Rashmika orcidid: 0000-0001-6641-2153 surname: Nawaratne fullname: Nawaratne, Rashmika email: b.nawaratne@latrobe.edu.au organization: Centre for Data Analytics and Cognition at La Trobe University, Melbourne, VIC, Australia – sequence: 2 givenname: Damminda surname: Alahakoon fullname: Alahakoon, Damminda email: d.alahakoon@latrobe.edu.au organization: Centre for Data Analytics and Cognition at La Trobe University, Melbourne, VIC, Australia – sequence: 3 givenname: Daswin orcidid: 0000-0003-3878-5969 surname: De Silva fullname: De Silva, Daswin email: d.desilva@latrobe.edu.au organization: Centre for Data Analytics and Cognition at La Trobe University, Melbourne, VIC, Australia – sequence: 4 givenname: Harsha surname: Kumara fullname: Kumara, Harsha email: h.kumara@latrobe.edu.au organization: Centre for Data Analytics and Cognition at La Trobe University, Melbourne, VIC, Australia – sequence: 5 givenname: Xinghuo orcidid: 0000-0001-8093-9787 surname: Yu fullname: Yu, Xinghuo email: xinghuo.yu@rmit.edu.au organization: School of Engineering, RMIT University, Melbourne, VIC, Australia
BookMark	eNp9kE1LAzEQhoMo-HkXvAQ8b83n7uZYitqCUtAVj0ucTWrKNqnZ1FJ_vbtUPHjwNDPwPjPMc4oOffAGoUtKRpQSdVPNZiNGqBoxJQshxQE6oUrQjBBJDvteSppxRvgxOu26JSG8IFydIJg6E3WEdwe6xdU2ZM8pGr3C9zFsnV_gZ9PabB4X2ruvYX7U6w6_uvSOq6h954wHg22IeLpZaY_HkNynSzv8ZCAsvEsu-HN0ZHXbmYufeoZe7m6ryTR7mN_PJuOHDJiiKWveclbonJbSKktAy8aANCAaKJuiaHLQBRXQCA7KytJaaYUwmlrOlWRWAz9D1_u96xg-NqZL9TJsou9P1kyInORlzlifIvsUxNB10dh6Hd1Kx11NST2orHuV9aCy_lHZI_kfBFzSw2spatf-B17tQWeM-b1TKsakIvwb7ZSEZg
CODEN	ITIICH
CitedBy_id	crossref_primary_10_1038_s41598_023_41231_0 crossref_primary_10_1109_ACCESS_2020_3000829 crossref_primary_10_1109_TII_2023_3264284 crossref_primary_10_1109_TII_2022_3192398 crossref_primary_10_1109_JIOT_2020_3033430 crossref_primary_10_1109_JIOT_2020_2966792 crossref_primary_10_1109_TII_2020_3037286 crossref_primary_10_1007_s11042_023_17748_x crossref_primary_10_1007_s44163_024_00123_6 crossref_primary_10_1155_2022_5681412 crossref_primary_10_3390_en17143572 crossref_primary_10_1109_ACCESS_2021_3111053 crossref_primary_10_1109_JSEN_2024_3511087 crossref_primary_10_1016_j_ins_2023_119717 crossref_primary_10_3390_app14156800 crossref_primary_10_1109_TNNLS_2022_3185638 crossref_primary_10_3390_make6020051 crossref_primary_10_3390_en17081935 crossref_primary_10_1007_s10796_020_10056_x
Cites_doi	10.1109/TMM.2012.2188783 10.1109/ICME.2018.8486447 10.1109/CVPR.2009.5206744 10.1145/1290082.1290111 10.1109/CVPR.2009.5206821 10.1109/TCSVT.2018.2889514 10.1109/TII.2013.2255616 10.3390/app7010110 10.3389/fnbot.2015.00003 10.1162/neco.1994.6.1.100 10.1109/TPAMI.2012.253 10.1007/978-3-642-56927-2 10.1109/CVPR.2005.177 10.1109/RTEICT.2017.8256851 10.1016/j.cviu.2016.04.009 10.1109/TPAMI.2016.2537337 10.1109/CVPR.2014.107 10.1016/j.neuron.2017.04.037 10.1109/IJCNN.2010.5596612 10.1109/TPAMI.2007.70711 10.1109/72.846732 10.1109/IJCNN.2017.7965886 10.1109/ICPR.2004.1334462 10.1109/IECON.2017.8216826
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/TII.2019.2957454
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering Architecture
EISSN	1941-0050
EndPage	7764
ExternalDocumentID	10_1109_TII_2019_2957454 8922590
Genre	orig-research
GrantInformation_xml	– fundername: La Trobe University Postgraduate Research Scholarship
GroupedDBID	0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ IFIPE IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c291t-db627a6185f9f0ca5dec5ec4dc8d77d6ca714cd43c9f58ff5f44ea1f33952fac3
IEDL.DBID	RIE
ISICitedReferencesCount	29
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000572631900050&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1551-3203
IngestDate	Mon Jun 30 10:21:15 EDT 2025 Tue Nov 18 21:52:55 EST 2025 Sat Nov 29 04:16:50 EST 2025 Wed Aug 27 02:31:57 EDT 2025
IsPeerReviewed	false
IsScholarly	true
Issue	12
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c291t-db627a6185f9f0ca5dec5ec4dc8d77d6ca714cd43c9f58ff5f44ea1f33952fac3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0003-3878-5969 0000-0001-6641-2153 0000-0001-8093-9787
PQID	2446068622
PQPubID	85507
PageCount	9
ParticipantIDs	proquest_journals_2446068622 ieee_primary_8922590 crossref_primary_10_1109_TII_2019_2957454 crossref_citationtrail_10_1109_TII_2019_2957454
PublicationCentury	2000
PublicationDate	2020-12-01
PublicationDateYYYYMMDD	2020-12-01
PublicationDate_xml	– month: 12 year: 2020 text: 2020-12-01 day: 01
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE transactions on industrial informatics
PublicationTitleAbbrev	TII
PublicationYear	2020
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref12 ref15 ref14 liu (ref1) 2013; 9 umakanthan (ref11) 2016 ref2 ref17 ref16 ref19 ref18 ref24 ref23 ref25 ref20 bergstra (ref26) 2012; 13 simonyan (ref4) 0 ref22 ref21 ref28 ref27 ref29 ref8 ref7 ref9 nawaratne (ref5) 2019 ref3 ref6 kong (ref10) 2018
References_xml	– ident: ref22 doi: 10.1109/TMM.2012.2188783 – ident: ref29 doi: 10.1109/ICME.2018.8486447 – ident: ref25 doi: 10.1109/CVPR.2009.5206744 – ident: ref14 doi: 10.1145/1290082.1290111 – ident: ref17 doi: 10.1109/CVPR.2009.5206821 – ident: ref21 doi: 10.1109/TCSVT.2018.2889514 – volume: 9 start-page: 1222 year: 2013 ident: ref1 article-title: Intelligent video systems and analytics: A survey publication-title: IEEE Trans Ind Informat doi: 10.1109/TII.2013.2255616 – ident: ref2 doi: 10.3390/app7010110 – ident: ref15 doi: 10.3389/fnbot.2015.00003 – year: 2019 ident: ref5 article-title: Spatiotemporal anomaly detection using deep learning for real-time video surveillance publication-title: IEEE Trans Ind Informat – ident: ref18 doi: 10.1162/neco.1994.6.1.100 – ident: ref12 doi: 10.1109/TPAMI.2012.253 – year: 2018 ident: ref10 article-title: Human action recognition and prediction: A survey – ident: ref19 doi: 10.1007/978-3-642-56927-2 – ident: ref16 doi: 10.1109/CVPR.2005.177 – ident: ref3 doi: 10.1109/RTEICT.2017.8256851 – ident: ref8 doi: 10.1016/j.cviu.2016.04.009 – ident: ref13 doi: 10.1109/TPAMI.2016.2537337 – ident: ref27 doi: 10.1109/CVPR.2014.107 – ident: ref9 doi: 10.1016/j.neuron.2017.04.037 – ident: ref20 doi: 10.1109/IJCNN.2010.5596612 – ident: ref23 doi: 10.1109/TPAMI.2007.70711 – ident: ref7 doi: 10.1109/72.846732 – ident: ref28 doi: 10.1109/IJCNN.2017.7965886 – year: 2016 ident: ref11 article-title: Human action recognition from video sequences – start-page: 568 year: 0 ident: ref4 article-title: Two-stream convolutional networks for action recognition in videos publication-title: Advances in Neural Information Processing Systems 27 – ident: ref24 doi: 10.1109/ICPR.2004.1334462 – ident: ref6 doi: 10.1109/IECON.2017.8216826 – volume: 13 start-page: 281 year: 2012 ident: ref26 article-title: Random search for hyper-parameter optimization publication-title: J Mach Learn Res
SSID	ssj0037039
Score	2.4252017
Snippet	The rapid growth in autonomous industrial environments has increased the need for intelligent video surveillance. As a predominant element of video...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	7756
SubjectTerms	Activity recognition Algorithms Architecture Decision making Deep learning Feature extraction Forgetting hierarchical learning Histograms Human activity recognition human activity recognition (HAR) Machine learning Moving object recognition neural networks Neurons Optical flow Self organizing maps Self-organizing feature maps Surveillance Video data Video surveillance
Title	Hierarchical Two-Stream Growing Self-Organizing Maps With Transience for Human Activity Recognition
URI	https://ieeexplore.ieee.org/document/8922590 https://www.proquest.com/docview/2446068622
Volume	16
WOSCitedRecordID	wos000572631900050&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1941-0050 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0037039 issn: 1551-3203 databaseCode: RIE dateStart: 20050101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8QwEA4qHvTgW1xf5OBFMG7TJk3nKOLroIiu6K2kkwQX1t3FXfXvm2TbRVEEby1MoPSbZGYyj4-QAwChueacoQ1pRlVZplFJxqs8Ra0cpsZFsgl1c1M8PcHtDDma9sJYa2PxmT0OjzGXbwb4Fq7K2gV47QMfoM8qlU96tZpTN_OaC3E2quQsS5OsSUkm0O5cXYUaLjhOQSohxTcTFDlVfhzE0bqcL__vu1bIUu1F0pMJ7KtkxvbXyOKX2YLrBC-7obc4Up30aOdjwEICWr_QCx94ewl6b3uO1a2Y4f1aD0f0sTt-ptF-xS1PvUtL4z0_PcEJzwS9a0qOBv0N8nB-1jm9ZDWjAsMU-JgZj4DSubfRDlyCWhqL0qIwWBilTO7R4QKNyBCcLJyTTgirucsykKnTmG2Suf6gb7cIrSDLFa9M4gpv5SEHi5XKlXd3hE20qlqk3fzkEutx44H1olfGsCOB0sNSBljKGpYWOZyuGE5Gbfwhux5gmMrVCLTIboNjWe_FUekdmDw2wqTbv6_aIQtpiKJjkcoumRu_vtk9Mo_v4-7odT-q2SchbNGH
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3daxQxEB9KLagPWq3i2Q_z4ItgepvdZLPzWIrtHW0PaU_s25KdJPTgvCu9q_77JrndQ7EIfduFCSz7m2RmMh8_gI-I0ggjBCcX04y6cdyQVlw0ZU5Ge8qtT2QTejSqrq_x6wZ8XvfCOOdS8Zk7jI8pl2_ndB-vyvoVBu3DEKA_UcGOlqture7cLYLuYpqOqgQv8qzokpIZ9sfDYaziwsMclZZK_mWEEqvKP0dxsi8nLx_3ZdvwovUj2dEK-Few4Wav4fkf0wV3gAaT2F2cyE6mbPxrzmMK2vxgpyH0DhLsyk09b5sx4_uFuV2w75PlDUsWLG16Fpxalm762RGtmCbYZVd0NJ-9gW8nX8bHA95yKnDKUSy5DRhoUwYr7dFnZJR1pBxJS5XV2pYBHyHJyoLQq8p75aV0RviiQJV7Q8Vb2JzNZ-4dsAaLUovGZr4Kdh5LdNToUgeHR7rM6KYH_e4n19QOHI-8F9M6BR4Z1gGWOsJSt7D04NN6xe1q2MZ_ZHciDGu5FoEe7HU41u1uXNTBhSlTK0z-_uFVH-DpYHxxXp8PR2e78CyPMXUqWdmDzeXdvduHLfq5nCzuDpLK_QZNJdTa
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Hierarchical+Two-Stream+Growing+Self-Organizing+Maps+With+Transience+for+Human+Activity+Recognition&rft.jtitle=IEEE+transactions+on+industrial+informatics&rft.au=Nawaratne%2C+Rashmika&rft.au=Alahakoon%2C+Damminda&rft.au=De+Silva%2C+Daswin&rft.au=Kumara%2C+Harsha&rft.date=2020-12-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=1551-3203&rft.eissn=1941-0050&rft.volume=16&rft.issue=12&rft.spage=7756&rft_id=info:doi/10.1109%2FTII.2019.2957454&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1551-3203&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1551-3203&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1551-3203&client=summon