Real-Time Human Action Anomaly Detection Through Two-Stream Spatial-Temporal Networks
Human Action Anomaly Detection is an advanced technology that leverages computer vision and machine learning to identify unusual or suspicious human activities in real-time video stream and sensor data. By continuously monitoring environments such as public spaces, workplaces, and residential areas,...
Uloženo v:
| Vydáno v: | IEEE access Ročník 13; s. 66774 - 66786 |
|---|---|
| Hlavní autoři: | , , , , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Piscataway
IEEE
2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Témata: | |
| ISSN: | 2169-3536, 2169-3536 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Human Action Anomaly Detection is an advanced technology that leverages computer vision and machine learning to identify unusual or suspicious human activities in real-time video stream and sensor data. By continuously monitoring environments such as public spaces, workplaces, and residential areas, this system can promptly detect and respond to potential threats and safety violations. The core components of this technology include data collection from video surveillance and sensors, preprocessing techniques for feature extraction, model training using normal action patterns, and real-time anomaly detection algorithms. Due to the hardware limitations in many industrial scenarios, we would like to explore the use of the CPU alone to detect abnormal human action. In this paper, we present a two-stream spatial-temporal transformer network for predicting human-object interactions in real-time. Initially, human skeleton information was extracted using a human pose detector. A human hand detector and tracker are utilized to localize the human hand and detect its joints in greater detail. Concurrently, an object detector was employed to spatially and semantically localize and classify the objects with which the worker in the image was interacting. With extracted information as input, we model human action anomaly detection in operations using both spatial and temporal dimensions. We validated our work with the composed network on our own collected dataset and experimentally proved that our work can significantly identify incorrect work steps in a work scenario. |
|---|---|
| AbstractList | Human Action Anomaly Detection is an advanced technology that leverages computer vision and machine learning to identify unusual or suspicious human activities in real-time video stream and sensor data. By continuously monitoring environments such as public spaces, workplaces, and residential areas, this system can promptly detect and respond to potential threats and safety violations. The core components of this technology include data collection from video surveillance and sensors, preprocessing techniques for feature extraction, model training using normal action patterns, and real-time anomaly detection algorithms. Due to the hardware limitations in many industrial scenarios, we would like to explore the use of the CPU alone to detect abnormal human action. In this paper, we present a two-stream spatial-temporal transformer network for predicting human-object interactions in real-time. Initially, human skeleton information was extracted using a human pose detector. A human hand detector and tracker are utilized to localize the human hand and detect its joints in greater detail. Concurrently, an object detector was employed to spatially and semantically localize and classify the objects with which the worker in the image was interacting. With extracted information as input, we model human action anomaly detection in operations using both spatial and temporal dimensions. We validated our work with the composed network on our own collected dataset and experimentally proved that our work can significantly identify incorrect work steps in a work scenario. |
| Author | Jiang, Zebin Hu, Hongbin Xu, Xiao Wu, Yuankai Qing, Cen Peng, Chuan Lin, Mao |
| Author_xml | – sequence: 1 givenname: Chuan surname: Peng fullname: Peng, Chuan organization: State Grid Sichuan Electric Power Corporation, Ziyang, China – sequence: 2 givenname: Zebin orcidid: 0009-0009-5599-3437 surname: Jiang fullname: Jiang, Zebin organization: Technical University of Munich, Munich, Germany – sequence: 3 givenname: Mao surname: Lin fullname: Lin, Mao organization: State Grid Sichuan Electric Power Corporation, Ziyang, China – sequence: 4 givenname: Hongbin surname: Hu fullname: Hu, Hongbin organization: State Grid Sichuan Electric Power Corporation, Ziyang, China – sequence: 5 givenname: Cen surname: Qing fullname: Qing, Cen organization: State Grid Sichuan Electric Power Corporation, Ziyang, China – sequence: 6 givenname: Yuankai orcidid: 0000-0003-0573-2609 surname: Wu fullname: Wu, Yuankai organization: Technical University of Munich, Munich, Germany – sequence: 7 givenname: Xiao orcidid: 0000-0002-4375-3884 surname: Xu fullname: Xu, Xiao email: xiao.xu@tum.de organization: Technical University of Munich, Munich, Germany |
| BookMark | eNp9UdFq2zAUFSODtV2_YHsw9NmZZFmW9RiydC2UDZb0WVzJ14lS28pkhdC_rzJ3UPZQvRxxOOdw7z2XZDb4AQn5wuicMaq-LZbL1Xo9L2gh5lxUVFL-gVwUrFI5F7yavfl_ItfjuKfp1YkS8oI8_kbo8o3rMbs79jBkCxudTzD4Hrrn7DtGnJjNLvjjdpdtTj5fx4DQZ-sDRHe2Y3_wAbrsJ8aTD0_jZ_KxhW7E61e8Io-3q83yLn_49eN-uXjIbVnzmBuDCUXFGlqUAplQtYRSQdEILK1hrVUVlkZK20gjqlIlDbXcGsHTSobyK3I_5TYe9voQXA_hWXtw-i_hw1ZDiM52qEGilGBlq4qmpKVQTUsVWlS1YcJUTcq6mbIOwf854hj13h_DkMbXnCleC1orkVRqUtngxzFgq62LcD5QDOA6zag-l6KnUvS5FP1aSvLy_7z_Jn7f9XVyOUR841BVOpziL-3HmcU |
| CODEN | IAECCG |
| CitedBy_id | crossref_primary_10_3390_agriculture15151609 |
| Cites_doi | 10.1109/ICME.2019.00169 10.1109/ICST47872.2019.9166366 10.1109/ICASSP39728.2021.9414222 10.1109/IROS55552.2023.10341409 10.1109/ICASSP.2018.8461595 10.1109/CTCEEC.2017.8455012 10.1109/TNNLS.2016.2582924 10.1109/WiSPNET.2017.8299805 10.1109/TPAMI.2019.2929257 10.1109/ICACRS58579.2023.10404670 10.1109/CAC48633.2019.8997171 10.1109/ACPR.2017.98 10.1109/IMCCC.2011.95 10.1109/TIP.2018.2818328 10.1109/ICME51207.2021.9428459 10.1109/CVPR52688.2022.01947 10.1109/ACCAI61061.2024.10602454 10.1016/j.neunet.2021.12.008 10.1109/AIC57670.2023.10263965 10.1109/LSP.2021.3128379 10.1109/IPRIA53572.2021.9483554 10.1109/ICACITE51222.2021.9404753 10.1109/DeSE60595.2023.10469231 10.1109/ACPR.2013.129 10.1109/ICCRE61448.2024.10589753 10.1109/ICACS55311.2023.10089764 10.1109/ICASSP40776.2020.9054456 10.1109/ICCE-China.2016.7849737 10.1109/IPAS50080.2020.9334942 10.1109/CISP-BMEI.2018.8633176 10.1109/CSAE.2011.5953287 10.1109/ACCAI61061.2024.10602098 10.1109/ICCV51070.2023.00955 10.1109/ICPAI51961.2020.00039 10.1109/ICoAC.2015.7562795 10.1109/ICASSP48485.2024.10445978 10.1109/ICASSP48485.2024.10445852 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025 |
| DBID | 97E ESBDL RIA RIE AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D DOA |
| DOI | 10.1109/ACCESS.2025.3560703 |
| DatabaseName | IEEE Xplore (IEEE) Open Access资源_IEL Journals IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Engineered Materials Abstracts METADEX Technology Research Database Materials Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional DOAJ Directory of Open Access Journals |
| DatabaseTitle | CrossRef Materials Research Database Engineered Materials Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace METADEX Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Materials Research Database |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2169-3536 |
| EndPage | 66786 |
| ExternalDocumentID | oai_doaj_org_article_a7e77ac7f92d40459df09ece98b15b6d 10_1109_ACCESS_2025_3560703 10965619 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Science and Technology Project of State Grid Sichuan Electric Power Corporation grantid: 521918230001 funderid: 10.13039/501100009578 |
| GroupedDBID | 0R~ 4.4 5VS 6IK 97E AAJGR ABAZT ABVLG ACGFS ADBBV AGSQL ALMA_UNASSIGNED_HOLDINGS BCNDV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD ESBDL GROUPED_DOAJ IPLJI JAVBF KQ8 M43 M~E O9- OCL OK1 RIA RIE RNS AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c483t-bbe483561d0245e15987a49a2d5e4cb1fc96e4b77cd7b56495e10c3cb53216b03 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 1 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001473151800025&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2169-3536 |
| IngestDate | Fri Oct 03 12:44:39 EDT 2025 Mon Jun 30 11:33:15 EDT 2025 Tue Nov 18 21:48:57 EST 2025 Sat Nov 29 08:01:57 EST 2025 Wed Aug 27 02:04:02 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| License | https://creativecommons.org/licenses/by/4.0/legalcode |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c483t-bbe483561d0245e15987a49a2d5e4cb1fc96e4b77cd7b56495e10c3cb53216b03 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0009-0009-5599-3437 0000-0003-0573-2609 0000-0002-4375-3884 |
| OpenAccessLink | https://ieeexplore.ieee.org/document/10965619 |
| PQID | 3193850895 |
| PQPubID | 4845423 |
| PageCount | 13 |
| ParticipantIDs | proquest_journals_3193850895 crossref_citationtrail_10_1109_ACCESS_2025_3560703 doaj_primary_oai_doaj_org_article_a7e77ac7f92d40459df09ece98b15b6d ieee_primary_10965619 crossref_primary_10_1109_ACCESS_2025_3560703 |
| PublicationCentury | 2000 |
| PublicationDate | 20250000 2025-00-00 20250101 2025-01-01 |
| PublicationDateYYYYMMDD | 2025-01-01 |
| PublicationDate_xml | – year: 2025 text: 20250000 |
| PublicationDecade | 2020 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE access |
| PublicationTitleAbbrev | Access |
| PublicationYear | 2025 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref35 ref12 ref34 ref15 ref37 ref36 ref31 ref30 ref11 ref33 ref10 ref32 ref2 ref1 Sophia (ref14) ref17 ref16 ref38 ref19 ref18 ref24 ref23 ref26 ref25 ref20 ref22 Huang (ref39) 2015 ref21 ref28 ref27 ref29 ref8 ref7 ref9 ref4 ref3 ref6 ref5 |
| References_xml | – ident: ref12 doi: 10.1109/ICME.2019.00169 – ident: ref34 doi: 10.1109/ICST47872.2019.9166366 – ident: ref9 doi: 10.1109/ICASSP39728.2021.9414222 – ident: ref18 doi: 10.1109/IROS55552.2023.10341409 – ident: ref1 doi: 10.1109/ICASSP.2018.8461595 – ident: ref3 doi: 10.1109/CTCEEC.2017.8455012 – ident: ref7 doi: 10.1109/TNNLS.2016.2582924 – ident: ref22 doi: 10.1109/WiSPNET.2017.8299805 – start-page: 70 volume-title: Proc. Int. Conf. Inventive Comput. Technol. (ICICT) ident: ref14 article-title: Human behaviour and abnormality detection using YOLO and Conv2D net – ident: ref28 doi: 10.1109/TPAMI.2019.2929257 – ident: ref31 doi: 10.1109/ICACRS58579.2023.10404670 – ident: ref24 doi: 10.1109/CAC48633.2019.8997171 – ident: ref25 doi: 10.1109/ACPR.2017.98 – ident: ref21 doi: 10.1109/IMCCC.2011.95 – ident: ref37 doi: 10.1109/TIP.2018.2818328 – year: 2015 ident: ref39 article-title: Bidirectional LSTM-CRF models for sequence tagging publication-title: arXiv:1508.01991 – ident: ref10 doi: 10.1109/ICME51207.2021.9428459 – ident: ref15 doi: 10.1109/CVPR52688.2022.01947 – ident: ref30 doi: 10.1109/ACCAI61061.2024.10602454 – ident: ref8 doi: 10.1016/j.neunet.2021.12.008 – ident: ref38 doi: 10.1109/AIC57670.2023.10263965 – ident: ref11 doi: 10.1109/LSP.2021.3128379 – ident: ref23 doi: 10.1109/IPRIA53572.2021.9483554 – ident: ref33 doi: 10.1109/ICACITE51222.2021.9404753 – ident: ref5 doi: 10.1109/DeSE60595.2023.10469231 – ident: ref19 doi: 10.1109/ACPR.2013.129 – ident: ref29 doi: 10.1109/ICCRE61448.2024.10589753 – ident: ref32 doi: 10.1109/ICACS55311.2023.10089764 – ident: ref2 doi: 10.1109/ICASSP40776.2020.9054456 – ident: ref17 doi: 10.1109/ICCE-China.2016.7849737 – ident: ref6 doi: 10.1109/IPAS50080.2020.9334942 – ident: ref36 doi: 10.1109/CISP-BMEI.2018.8633176 – ident: ref4 doi: 10.1109/CSAE.2011.5953287 – ident: ref13 doi: 10.1109/ACCAI61061.2024.10602098 – ident: ref16 doi: 10.1109/ICCV51070.2023.00955 – ident: ref26 doi: 10.1109/ICPAI51961.2020.00039 – ident: ref35 doi: 10.1109/ICoAC.2015.7562795 – ident: ref27 doi: 10.1109/ICASSP48485.2024.10445978 – ident: ref20 doi: 10.1109/ICASSP48485.2024.10445852 |
| SSID | ssj0000816957 |
| Score | 2.3540182 |
| Snippet | Human Action Anomaly Detection is an advanced technology that leverages computer vision and machine learning to identify unusual or suspicious human activities... |
| SourceID | doaj proquest crossref ieee |
| SourceType | Open Website Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 66774 |
| SubjectTerms | action recognition Algorithms Anomalies Anomaly detection Behavioral sciences Computational modeling Computer vision Data collection Deep learning Detectors Feature extraction human-object interaction Long short term memory Machine learning Real time Real-time systems real-world infrastructure Residential areas Sensors Skeleton Transformers Video data Workplaces |
| SummonAdditionalLinks | – databaseName: DOAJ Directory of Open Access Journals dbid: DOA link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LaxsxEBYl5NAcQh8pdZsGHXLsNnqsVquj6yT0EEwIDuQmJO0IAo4TbKel_76jh4Mh0FxyWhAS2h3Njr7R4_sIOZY9w3IZ08q9atrgY-OkcE2AyBk4BjFkyvwLPZ32NzfmckvqK50JK_TAxXAnToPWLuhoxNAi_jBDZAYCmN5z5bshRV-mzVYylWNwzzujdKUZ4sycjCcT_CJMCIX6IXGa1xuZrDoVZcb-KrHyLC7nyeb8HdmvKJGOy9u9J29g8YHsbXEHfiTXVwjxmnSDg-aFeDrONxQo5vN3bv6XnsIaSsmsSPHQ2Z_7Jm1CuzuahIhvU_NCTDWn03IafHVArs_PZpNfTdVIaELby3XjPeATQdCQ9lABwUmvXWucGBSg5XkMpoPWax0G7VWH6RBwFmTwSgreeSY_kZ3F_QI-EypaLXwXhVaCtTxEPyBWGJQHlUT8JIyI2JjLhkognnQs5jYnEszYYmObbGyrjUfk-1Ojh8Kf8f_qP9M4PFVN5Ne5AF3CVpewL7nEiBykUdzqzyBq5WZEDjfDauufurIYgmSPKNWoL6_R91fyNn1PWaQ5JDvr5SN8I7vh9_p2tTzKTvoP9Dno8g priority: 102 providerName: Directory of Open Access Journals |
| Title | Real-Time Human Action Anomaly Detection Through Two-Stream Spatial-Temporal Networks |
| URI | https://ieeexplore.ieee.org/document/10965619 https://www.proquest.com/docview/3193850895 https://doaj.org/article/a7e77ac7f92d40459df09ece98b15b6d |
| Volume | 13 |
| WOSCitedRecordID | wos001473151800025&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAON databaseName: DOAJ Directory of Open Access Journals customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: DOA dateStart: 20130101 isFulltext: true titleUrlDefault: https://www.doaj.org/ providerName: Directory of Open Access Journals – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2169-3536 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000816957 issn: 2169-3536 databaseCode: M~E dateStart: 20130101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Nb9QwELWg4gAHPotYKJUPHElJ7DiOj8vSigOsENpKvVn-GEuVtruouwVx4bczY3tXlRBIXJLIshUnL45nxp73GHsjxxbLZaLIvWr64FPjpHBNgNS14FpIIVPmf9Lz-XhxYb7UZPWcCwMAefMZnNBlXsuP63BDoTIc4QbNDyL5vKv1UJK19gEVUpAwSldmIaz6bjqb4UOgDyjUicSZXe-Usersk0n6q6rKH7_iPL-cPfrPnj1mD6shyacF-SfsDqyesge36AWfsfOvaAU2lOTBc6yeT3MSA0eX_8otf_IPsIVSsihqPXzxY93QOrW74qRVfEnNC3fVks_LhvHNITs_O13MPjZVRqEJ_Si3jfeAZ-xdpGVWQPtl1K43TkQFCE6Xghmg91qHqL0a0GOCrg0yeCVFN_hWPmcHq_UKXjAuei38kIRWou27kHxEcyIqD4p0_iRMmNi9XhsqxzhJXSxt9jVaYwsmljCxFZMJe7tv9K1QbPy7-nvCbV-V-LFzAQJi63CzToPWLuhkROzRajUxtQYCmNF3yg9xwg4JxFv3K_hN2NHuM7B1MG8s_qXkiIasUS__0uwVu09dLKGZI3awvb6B1-xe-L693FwfZz8fj59_nR7nb_Y3ruHnzg |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1daxQxFA2lFtQHW7Wlq1Xz4KNTZ5LJZPK4rpYW10VkC30L-biBwna3dLeW_ntvPnYpFAWfZggJk5mTTO69yT2HkI-8r7Gchxi5F1XrbKgMZ6ZyEJoaTA3BJcr8sZxM-osL9bMkq6dcGABIh8_gON6mvXy_cLcxVIYzXKH5EUk-n4gWHZ-crrUJqUQNCSVk4RbCyp-HoxG-BnqBTBxzXNvlWhurrD-Jpr_oqjz6GacV5mT3P_u2R14UU5IOM_YvyRbMX5HnDwgGX5PzX2gHVjHNg6ZoPR2mNAaKTv-Vmd3Tr7CCXDLNej10ereo4k61uaJRrfgyNs_sVTM6yUfGl_vk_OTbdHRaFSGFyrU9X1XWAl6xdz5utAJaML00rTLMC0B4muBUB62V0nlpRYc-EzS1484KzprO1vyAbM8XczgklLWS2S4wKVjdNi5YjwaFFxZEVPrjMCBs_Xm1KyzjUexippO3USudMdERE10wGZBPm0bXmWTj39W_RNw2VSNDdipAQHSZcNpIkNI4GRTzLdqtyodagQPV20bYzg_IfgTxwfMyfgNytB4Gukznpcb_FO_RlFXizV-afSBPT6c_xnp8Nvn-ljyL3c2BmiOyvbq5hXdkx_1eXS5v3qcx-wf2c-jv |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Real-Time+Human+Action+Anomaly+Detection+Through+Two-Stream+Spatial-Temporal+Networks&rft.jtitle=IEEE+access&rft.au=Peng%2C+Chuan&rft.au=Jiang%2C+Zebin&rft.au=Lin%2C+Mao&rft.au=Hu%2C+Hongbin&rft.date=2025&rft.pub=IEEE&rft.eissn=2169-3536&rft.volume=13&rft.spage=66774&rft.epage=66786&rft_id=info:doi/10.1109%2FACCESS.2025.3560703&rft.externalDocID=10965619 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2169-3536&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2169-3536&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2169-3536&client=summon |