M2S2: A Multimodal Sensor System for Remote Animal Motion Capture in the Wild
Capturing animal locomotion in the wild is far more challenging than in controlled laboratory settings. Wildlife subjects move unpredictably, and issues, such as scaling, occlusion, lighting changes, and the lack of ground truth data, make motion capture difficult. Unlike human biomechanics, where m...
Uloženo v:
| Vydáno v: | IEEE sensors letters Ročník 9; číslo 4; s. 1 - 4 |
|---|---|
| Hlavní autoři: | , , , , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
01.04.2025
|
| Témata: | |
| ISSN: | 2475-1472, 2475-1472 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Capturing animal locomotion in the wild is far more challenging than in controlled laboratory settings. Wildlife subjects move unpredictably, and issues, such as scaling, occlusion, lighting changes, and the lack of ground truth data, make motion capture difficult. Unlike human biomechanics, where machine learning thrives with annotated datasets, such resources are scarce for wildlife. Multimodal sensing offers a solution by combining the strengths of various sensors, such as Light Detection and Ranging {LiDAR) and thermal cameras, to compensate for individual sensor limitations. In addition, some sensors, like LiDAR, can provide training data for monocular pose estimation models. We introduce a multimodal sensor system (M2S2) for capturing animal motion in the wild. M2S2 integrates RGB, depth, thermal, event, LiDAR, and acoustic sensors to overcome challenges like synchronization and calibration. We showcase its application with data from cheetahs, offering a new resource for advancing sensor fusion algorithms in wildlife motion capture. |
|---|---|
| AbstractList | Capturing animal locomotion in the wild is far more challenging than in controlled laboratory settings. Wildlife subjects move unpredictably, and issues, such as scaling, occlusion, lighting changes, and the lack of ground truth data, make motion capture difficult. Unlike human biomechanics, where machine learning thrives with annotated datasets, such resources are scarce for wildlife. Multimodal sensing offers a solution by combining the strengths of various sensors, such as Light Detection and Ranging {LiDAR) and thermal cameras, to compensate for individual sensor limitations. In addition, some sensors, like LiDAR, can provide training data for monocular pose estimation models. We introduce a multimodal sensor system (M2S2) for capturing animal motion in the wild. M2S2 integrates RGB, depth, thermal, event, LiDAR, and acoustic sensors to overcome challenges like synchronization and calibration. We showcase its application with data from cheetahs, offering a new resource for advancing sensor fusion algorithms in wildlife motion capture. |
| Author | Paine, Stephen Bowden, Nicholas Amayo, Paul Maswoswere, Gerald Markham, Andrew Patel, Amir Vally, Azraa |
| Author_xml | – sequence: 1 givenname: Azraa orcidid: 0000-0002-0117-6943 surname: Vally fullname: Vally, Azraa email: vllazr002@myuct.ac.za organization: Department of Electrical Engineering, University of Cape Town, Cape Town, South Africa – sequence: 2 givenname: Gerald orcidid: 0009-0005-1444-5953 surname: Maswoswere fullname: Maswoswere, Gerald email: mswger001@myuct.ac.za organization: Department of Electrical Engineering, University of Cape Town, Cape Town, South Africa – sequence: 3 givenname: Nicholas surname: Bowden fullname: Bowden, Nicholas organization: Department of Electrical Engineering, University of Cape Town, Cape Town, South Africa – sequence: 4 givenname: Stephen orcidid: 0000-0001-8621-7005 surname: Paine fullname: Paine, Stephen organization: Department of Electrical Engineering, University of Cape Town, Cape Town, South Africa – sequence: 5 givenname: Paul orcidid: 0000-0002-6681-8230 surname: Amayo fullname: Amayo, Paul organization: Department of Electrical Engineering, University of Cape Town, Cape Town, South Africa – sequence: 6 givenname: Andrew orcidid: 0000-0001-5716-3941 surname: Markham fullname: Markham, Andrew organization: Department of Computer Science, University of Oxford, Oxford, U.K – sequence: 7 givenname: Amir orcidid: 0000-0002-2344-4179 surname: Patel fullname: Patel, Amir organization: Department of Electrical Engineering, University of Cape Town, Cape Town, South Africa |
| BookMark | eNpNkM1Kw0AUhQepYK19AXExL5A6904yM3FXSv2BRsEoLkPS3MFIkimZ6aJvb6pddHUPXL7D4btmk971xNgtiAWASO83-fo1X6DAZCGTGFHKCzbFWCcRxBonZ_mKzb3_EUKAQS2kmLIswxwf-JJn-zY0navLlufUezfw_OADddyO8Z06F4gv-6Yb_5kLjev5qtyF_UC86Xn4Jv7VtPUNu7Rl62l-ujP2-bj-WD1Hm7enl9VyE20RIUSkDGiFilCDqYyqSFgEMtsUgGoRp6WurUKTlClJELWA1FbC2nFxUspKyRnD_97t4LwfyBa7YZw2HAoQxdFJ8eekODopTk5G6O4faojoDDBGo1TyFzLKXaA |
| CODEN | ISLECD |
| Cites_doi | 10.1109/CVPRW53098.2021.00155 10.1007/978-3-030-58621-8_12 10.1117/12.829094 10.1109/JSTSP.2021.3061270 10.3389/fnbeh.2021.637806 10.3390/s23073479 10.1109/CVPR.2018.00790 10.1007/978-3-030-58523-5_1 10.1109/ICCV.2019.00546 10.1109/CVPR42600.2020.00836 10.1109/CVPRW59228.2023.00022 10.1038/s41592-021-01106-6 10.1109/IROS.2018.8593660 10.1109/CVPR42600.2020.00502 10.1109/WACV48630.2021.00190 10.1109/ICCV51070.2023.00835 10.1007/s11263-022-01714-5 10.1109/ACCESS.2023.3264714 10.1007/978-3-319-08651-4_1 10.3390/electronics12163423 10.3389/fnbeh.2020.581154 10.1109/ICRA48506.2021.9561338 10.1038/s41467-020-18441-5 10.1145/3560905.3567770 10.1038/s42256-022-00520-5 |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION |
| DOI | 10.1109/LSENS.2025.3542233 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISSN | 2475-1472 |
| EndPage | 4 |
| ExternalDocumentID | 10_1109_LSENS_2025_3542233 10887236 |
| Genre | orig-research |
| GroupedDBID | 0R~ 6IK 97E AAJGR AASAJ AAWTH ABAZT ABJNI ABQJQ ABVLG ACGFS AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF OCL RIA RIE AAYXX CITATION |
| ID | FETCH-LOGICAL-c221t-e6817626e2718b86be0f21e8c911ed049a7df6285a9e310d019fb0ff0305a3b63 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 1 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001445056900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2475-1472 |
| IngestDate | Sat Nov 29 08:09:15 EST 2025 Wed Aug 27 01:47:41 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 4 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c221t-e6817626e2718b86be0f21e8c911ed049a7df6285a9e310d019fb0ff0305a3b63 |
| ORCID | 0000-0002-0117-6943 0000-0001-8621-7005 0000-0002-2344-4179 0000-0002-6681-8230 0009-0005-1444-5953 0000-0001-5716-3941 |
| PageCount | 4 |
| ParticipantIDs | crossref_primary_10_1109_LSENS_2025_3542233 ieee_primary_10887236 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-04-01 |
| PublicationDateYYYYMMDD | 2025-04-01 |
| PublicationDate_xml | – month: 04 year: 2025 text: 2025-04-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | IEEE sensors letters |
| PublicationTitleAbbrev | LSENS |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| References | ref13 ref15 Feti (ref34) 2012 ref14 ref31 ref11 ref33 ref10 An (ref8) 2022 ref2 ref1 ref17 ref16 ref19 ref18 (ref32) 2024 ref24 ref23 Zanfir (ref26) 2023 ref25 ref20 Ci (ref28) 2022 ref22 ref21 ref27 (ref29) 2023 ref7 Yang (ref12) 2022 (ref30) 2023 ref9 ref4 ref6 ref5 Yang (ref3) 2023; 36 |
| References_xml | – ident: ref33 doi: 10.1109/CVPRW53098.2021.00155 – start-page: 17301 volume-title: Proc. Adv. Neural Inf. Process. Syst. year: 2022 ident: ref12 article-title: APT-36K: A large-scale benchmark for animal pose estimation and tracking – ident: ref17 doi: 10.1007/978-3-030-58621-8_12 – ident: ref15 doi: 10.1117/12.829094 – ident: ref4 doi: 10.1109/JSTSP.2021.3061270 – ident: ref14 doi: 10.3389/fnbeh.2021.637806 – start-page: 1114 volume-title: Proc. Conf. Robot Learn. year: 2023 ident: ref26 article-title: HUM3DIL: Semi-supervised multi-modal 3D human pose estimation for autonomous driving – ident: ref24 doi: 10.3390/s23073479 – ident: ref27 doi: 10.1109/CVPR.2018.00790 – ident: ref11 doi: 10.1007/978-3-030-58523-5_1 – ident: ref21 doi: 10.1109/ICCV.2019.00546 – start-page: 1752 volume-title: Proc. 35th Int. Conv. MIPRO year: 2012 ident: ref34 article-title: The procedure of a camera calibration using camera calibration toolbox for MATLAB – volume-title: LiDAR camera calibrator app year: 2024 ident: ref32 – ident: ref23 doi: 10.1109/CVPR42600.2020.00836 – volume-title: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops year: 2022 ident: ref28 article-title: The sound of motion: Multimodal horse motion estimation from video and audio – ident: ref5 doi: 10.1109/CVPRW59228.2023.00022 – ident: ref13 doi: 10.1038/s41592-021-01106-6 – year: 2023 ident: ref29 article-title: ROS 2: Robot operating system 2 – year: 2023 ident: ref30 article-title: eCAL: Efficient communication abstraction layer – volume: 36 start-page: 18756 volume-title: Proc. Adv. Neural Inf. Process. Syst. year: 2023 ident: ref3 article-title: MM-Fi: Multi-modal non-intrusive 4D human dataset for versatile wireless sensing – ident: ref31 doi: 10.1109/IROS.2018.8593660 – ident: ref25 doi: 10.1109/CVPR42600.2020.00502 – ident: ref10 doi: 10.1109/WACV48630.2021.00190 – start-page: 27414 volume-title: Proc. Adv. Neural Inf. Process. Syst. year: 2022 ident: ref8 article-title: mRI: Multi-modal 3D human pose estimation dataset using mmWave, RGB-D, and inertial sensors – ident: ref18 doi: 10.1109/ICCV51070.2023.00835 – ident: ref19 doi: 10.1007/s11263-022-01714-5 – ident: ref1 doi: 10.1109/ACCESS.2023.3264714 – ident: ref22 doi: 10.1007/978-3-319-08651-4_1 – ident: ref7 doi: 10.3390/electronics12163423 – ident: ref9 doi: 10.3389/fnbeh.2020.581154 – ident: ref20 doi: 10.1109/ICRA48506.2021.9561338 – ident: ref16 doi: 10.1038/s41467-020-18441-5 – ident: ref6 doi: 10.1145/3560905.3567770 – ident: ref2 doi: 10.1038/s42256-022-00520-5 |
| SSID | ssj0001827030 |
| Score | 2.2944179 |
| Snippet | Capturing animal locomotion in the wild is far more challenging than in controlled laboratory settings. Wildlife subjects move unpredictably, and issues, such... |
| SourceID | crossref ieee |
| SourceType | Index Database Publisher |
| StartPage | 1 |
| SubjectTerms | Animals Calibration Cameras Laser radar Motion capture Pose estimation Radar sensor fusion Sensor systems Sensors Synchronization Thermal sensors |
| Title | M2S2: A Multimodal Sensor System for Remote Animal Motion Capture in the Wild |
| URI | https://ieeexplore.ieee.org/document/10887236 |
| Volume | 9 |
| WOSCitedRecordID | wos001445056900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 2475-1472 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001827030 issn: 2475-1472 databaseCode: RIE dateStart: 20170101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELZoxcDCQxRRXvLAhlISJ_GDrapaMdAKEZC6RXF8ljo0qUrL7-fspKIMDGxR4kjRd7m7z_Z9PkLuCyuTwnBkbq7KIRGhCZS0OogSbUsdWsVL6ZtNiNlMzufqtRWrey0MAPjiMxi4S7-Xb-py65bK0MPRJVjMO6QjBG_EWj8LKpK5v3cnjAnV40s2nmU4BWTpIE4TzIPxr-Sz103FJ5PJyT8_45Qct6yRDhszn5EDqM7JdMoy9kSH1Itol7XBERnOSus1bc4hp0hI6RugMYAOq8USn0990x46KlZu54AuKooEkGJoMD3yMRm_j56DtjtCUDIWbQLgMsJIxoFhetGSawgti0CWGL7AIPEvhLFOIFkoQA5nkMtZBN86Dy9izeML0q3qCi4JFaVSGiBNNSZ05Bs6FlZCZAsJWsk46ZOHHWz5qjkEI_eTh1DlHuTcgZy3IPdJz2G2N7KB6-qP-9fkyL3elMPckO5mvYVbclh-bRaf6ztv5G9riaTj |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELagIMHCQxRRnh7YUErivGy2CrUqookQLVK3KI7PUocmVWn5_ZydVJSBgS1KrCj6Lnf32b7PR8h9rnmQqwiZm6lyCGJXOYJr6XiB1IV0tYgKbptNxGnKp1Px1ojVrRYGAGzxGXTNpd3LV1WxNktl6OHoEsyPdsmeaZ3VyLV-llQ4M__vRhrjisfRuJ-OcRLIwq4fBpgJ_V_pZ6ufik0ng-N_fsgJOWp4I-3Vhj4lO1CekSRhY_ZEe9TKaOeVwhFjnJdWS1qfRE6RktJ3QHMA7ZWzOT5PbNse-pwvzN4BnZUUKSDF4KDa5GPQnzwPnaY_glMw5q0ciLiHsSwChglG8kiCq5kHvMAABgqpfx4rbSSSuQBkcQrZnEb4tfHx3JeRf05aZVXCBaFxIYQECEOJKR0Zh_RjzcHTOQcpuB90yMMGtmxRH4OR2emDKzILcmZAzhqQO6RtMNsaWcN1-cf9O3IwnCSjbPSSvl6RQ_OqujjmmrRWyzXckP3iazX7XN5ag38DNdioLA |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=M2S2%3A+A+Multimodal+Sensor+System+for+Remote+Animal+Motion+Capture+in+the+Wild&rft.jtitle=IEEE+sensors+letters&rft.au=Vally%2C+Azraa&rft.au=Maswoswere%2C+Gerald&rft.au=Bowden%2C+Nicholas&rft.au=Paine%2C+Stephen&rft.date=2025-04-01&rft.issn=2475-1472&rft.eissn=2475-1472&rft.volume=9&rft.issue=4&rft.spage=1&rft.epage=4&rft_id=info:doi/10.1109%2FLSENS.2025.3542233&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_LSENS_2025_3542233 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2475-1472&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2475-1472&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2475-1472&client=summon |