Zobrazit v EDS

3D hand pose and mesh estimation via a generic Topology-aware Transformer model.

Uloženo v:
Podrobná bibliografie
Název: 3D hand pose and mesh estimation via a generic Topology-aware Transformer model.
Autoři: Shaoqi Yu, Yintong Wang, Lili Chen, Xiaolin Zhang, Jiamao Li
Zdroj: Frontiers in Neurorobotics; 2024, p1-15, 15p
Témata: TRANSFORMER models, HUMAN-robot interaction
Abstrakt: In Human-Robot Interaction (HRI), accurate 3D hand pose and mesh estimation hold critical importance. However, inferring reasonable and accurate poses in severe self-occlusion and high self-similarity remains an inherent challenge. In order to alleviate the ambiguity caused by invisible and similar joints during HRI, we propose a new Topology-aware Transformer network named HandGCNFormer with depth image as input, incorporating prior knowledge of hand kinematic topology into the network while modeling long-range contextual information. Specifically, we propose a novel Graphformer decoder with an additional Node-offset Graph Convolutional layer (NoffGConv). The Graphformer decoder optimizes the synergy between the Transformer and GCN, capturing long-range dependencies and local topological connections between joints. On top of that, we replace the standard MLP prediction head with a novel Topology-aware head to better exploit local topological constraints for more reasonable and accurate poses. Our method achieves state-of-the-art 3D hand pose estimation performance on four challenging datasets, including Hands2017, NYU, ICVL, and MSRA. To further demonstrate the effectiveness and scalability of our proposed Graphformer Decoder and Topology aware head, we extend our framework to HandGCNFormer-Mesh for the 3D hand mesh estimation task. The extended framework efficiently integrates a shape regressor with the original Graphformer Decoder and Topology aware head, producing Mano parameters. The results on the HO-3D dataset, which contains various and challenging occlusions, show that our HandGCNFormer-Mesh achieves competitive results compared to previous state-of-the-art 3D hand mesh estimation methods. [ABSTRACT FROM AUTHOR]
Copyright of Frontiers in Neurorobotics is the property of Frontiers Media S.A. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Databáze: Biomedical Index
FullText Text:
  Availability: 0
CustomLinks:
  – Url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&db=pmc&term=1662-5218[TA]+AND+1[PG]+AND+2024[PDAT]
    Name: FREE - PubMed Central (ISSN based link)
    Category: fullText
    Text: Full Text
    Icon: https://imageserver.ebscohost.com/NetImages/iconPdf.gif
    MouseOverText: Check this PubMed for the article full text.
  – Url: https://resolver.ebscohost.com/openurl?sid=EBSCO:edm&genre=article&issn=16625218&ISBN=&volume=&issue=&date=20240517&spage=1&pages=1-15&title=Frontiers in Neurorobotics&atitle=3D%20hand%20pose%20and%20mesh%20estimation%20via%20a%20generic%20Topology-aware%20Transformer%20model.&aulast=Shaoqi%20Yu&id=DOI:10.3389/fnbot.2024.1395652
    Name: Full Text Finder
    Category: fullText
    Text: Full Text Finder
    Icon: https://imageserver.ebscohost.com/branding/images/FTF.gif
    MouseOverText: Full Text Finder
  – Url: https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=EBSCO&SrcAuth=EBSCO&DestApp=WOS&ServiceName=TransferToWoS&DestLinkType=GeneralSearchSummary&Func=Links&author=Yu%20S
    Name: ISI
    Category: fullText
    Text: Nájsť tento článok vo Web of Science
    Icon: https://imagesrvr.epnet.com/ls/20docs.gif
    MouseOverText: Nájsť tento článok vo Web of Science
Header DbId: edm
DbLabel: Biomedical Index
An: 177327839
RelevancyScore: 983
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 983.421691894531
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: 3D hand pose and mesh estimation via a generic Topology-aware Transformer model.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Shaoqi+Yu%22">Shaoqi Yu</searchLink><br /><searchLink fieldCode="AR" term="%22Yintong+Wang%22">Yintong Wang</searchLink><br /><searchLink fieldCode="AR" term="%22Lili+Chen%22">Lili Chen</searchLink><br /><searchLink fieldCode="AR" term="%22Xiaolin+Zhang%22">Xiaolin Zhang</searchLink><br /><searchLink fieldCode="AR" term="%22Jiamao+Li%22">Jiamao Li</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: Frontiers in Neurorobotics; 2024, p1-15, 15p
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22TRANSFORMER+models%22">TRANSFORMER models</searchLink><br /><searchLink fieldCode="DE" term="%22HUMAN-robot+interaction%22">HUMAN-robot interaction</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: In Human-Robot Interaction (HRI), accurate 3D hand pose and mesh estimation hold critical importance. However, inferring reasonable and accurate poses in severe self-occlusion and high self-similarity remains an inherent challenge. In order to alleviate the ambiguity caused by invisible and similar joints during HRI, we propose a new Topology-aware Transformer network named HandGCNFormer with depth image as input, incorporating prior knowledge of hand kinematic topology into the network while modeling long-range contextual information. Specifically, we propose a novel Graphformer decoder with an additional Node-offset Graph Convolutional layer (NoffGConv). The Graphformer decoder optimizes the synergy between the Transformer and GCN, capturing long-range dependencies and local topological connections between joints. On top of that, we replace the standard MLP prediction head with a novel Topology-aware head to better exploit local topological constraints for more reasonable and accurate poses. Our method achieves state-of-the-art 3D hand pose estimation performance on four challenging datasets, including Hands2017, NYU, ICVL, and MSRA. To further demonstrate the effectiveness and scalability of our proposed Graphformer Decoder and Topology aware head, we extend our framework to HandGCNFormer-Mesh for the 3D hand mesh estimation task. The extended framework efficiently integrates a shape regressor with the original Graphformer Decoder and Topology aware head, producing Mano parameters. The results on the HO-3D dataset, which contains various and challenging occlusions, show that our HandGCNFormer-Mesh achieves competitive results compared to previous state-of-the-art 3D hand mesh estimation methods. [ABSTRACT FROM AUTHOR]
– Name: Abstract
  Label:
  Group: Ab
  Data: <i>Copyright of Frontiers in Neurorobotics is the property of Frontiers Media S.A. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://erproxy.cvtisr.sk/sfx/access?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edm&AN=177327839
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.3389/fnbot.2024.1395652
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 15
        StartPage: 1
    Subjects:
      – SubjectFull: TRANSFORMER models
        Type: general
      – SubjectFull: HUMAN-robot interaction
        Type: general
    Titles:
      – TitleFull: 3D hand pose and mesh estimation via a generic Topology-aware Transformer model.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Shaoqi Yu
      – PersonEntity:
          Name:
            NameFull: Yintong Wang
      – PersonEntity:
          Name:
            NameFull: Lili Chen
      – PersonEntity:
          Name:
            NameFull: Xiaolin Zhang
      – PersonEntity:
          Name:
            NameFull: Jiamao Li
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 17
              M: 05
              Text: 2024
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 16625218
          Titles:
            – TitleFull: Frontiers in Neurorobotics
              Type: main
ResultId 1