Multi-Keypoint Affordance Representation for Functional Dexterous Grasping
Uloženo v:
| Název: | Multi-Keypoint Affordance Representation for Functional Dexterous Grasping |
|---|---|
| Autoři: | Fan Yang, Dongsheng Luo, Wenrui Chen, Jiacheng Lin, Junjie Cai, Kailun Yang, Zhiyong Li, Yaonan Wang |
| Zdroj: | IEEE Robotics and Automation Letters. 10:10306-10313 |
| Publication Status: | Preprint |
| Informace o vydavateli: | Institute of Electrical and Electronics Engineers (IEEE), 2025. |
| Rok vydání: | 2025 |
| Témata: | FOS: Computer and information sciences, Computer Vision and Pattern Recognition (cs.CV), Image and Video Processing (eess.IV), FOS: Electrical engineering, electronic engineering, information engineering, Image and Video Processing, Robotics, Computer Vision and Pattern Recognition, Robotics (cs.RO) |
| Popis: | Functional dexterous grasping requires precise hand-object interaction, going beyond simple gripping. Existing affordance-based methods primarily predict coarse interaction regions and cannot directly constrain the grasping posture, leading to a disconnection between visual perception and manipulation. To address this issue, we propose a multi-keypoint affordance representation for functional dexterous grasping, which directly encodes task-driven grasp configurations by localizing functional contact points. Our method introduces Contact-guided Multi-Keypoint Affordance (CMKA), leveraging human grasping experience images for weak supervision combined with Large Vision Models for fine affordance feature extraction, achieving generalization while avoiding manual keypoint annotations. Additionally, we present a Keypoint-based Grasp matrix Transformation (KGT) method, ensuring spatial consistency between hand keypoints and object contact points, thus providing a direct link between visual perception and dexterous grasping actions. Experiments on public real-world FAH datasets, IsaacGym simulation, and challenging robotic tasks demonstrate that our method significantly improves affordance localization accuracy, grasp consistency, and generalization to unseen tools and tasks, bridging the gap between visual affordance learning and dexterous robotic manipulation. The source code and demo videos are publicly available at https://github.com/PopeyePxx/MKA. Accepted to IEEE Robotics and Automation Letters (RA-L). The source code and demo videos are publicly available at https://github.com/PopeyePxx/MKA |
| Druh dokumentu: | Article |
| ISSN: | 2377-3774 |
| DOI: | 10.1109/lra.2025.3601041 |
| DOI: | 10.48550/arxiv.2502.20018 |
| Přístupová URL adresa: | http://arxiv.org/abs/2502.20018 |
| Rights: | IEEE Copyright arXiv Non-Exclusive Distribution |
| Přístupové číslo: | edsair.doi.dedup.....4d4c7444a4c17a1417da2da225d75828 |
| Databáze: | OpenAIRE |
Nájsť tento článok vo Web of Science | Abstrakt: | Functional dexterous grasping requires precise hand-object interaction, going beyond simple gripping. Existing affordance-based methods primarily predict coarse interaction regions and cannot directly constrain the grasping posture, leading to a disconnection between visual perception and manipulation. To address this issue, we propose a multi-keypoint affordance representation for functional dexterous grasping, which directly encodes task-driven grasp configurations by localizing functional contact points. Our method introduces Contact-guided Multi-Keypoint Affordance (CMKA), leveraging human grasping experience images for weak supervision combined with Large Vision Models for fine affordance feature extraction, achieving generalization while avoiding manual keypoint annotations. Additionally, we present a Keypoint-based Grasp matrix Transformation (KGT) method, ensuring spatial consistency between hand keypoints and object contact points, thus providing a direct link between visual perception and dexterous grasping actions. Experiments on public real-world FAH datasets, IsaacGym simulation, and challenging robotic tasks demonstrate that our method significantly improves affordance localization accuracy, grasp consistency, and generalization to unseen tools and tasks, bridging the gap between visual affordance learning and dexterous robotic manipulation. The source code and demo videos are publicly available at https://github.com/PopeyePxx/MKA.<br />Accepted to IEEE Robotics and Automation Letters (RA-L). The source code and demo videos are publicly available at https://github.com/PopeyePxx/MKA |
|---|---|
| ISSN: | 23773774 |
| DOI: | 10.1109/lra.2025.3601041 |