Hand Keypoint-Based CNN for SIBI Sign Language Recognition

SIBI is less widely adopted, and the lack of an efficient recognition system limits its accessibility. SIBI gestures often involve subtle hand movements and complex finger configurations, requiring precise feature extraction and classification techniques. This study addresses these issues using a Ha...

Full description

Saved in:
Bibliographic Details
Published in:International Journal of Robotics and Control Systems Vol. 5; no. 2; pp. 813 - 829
Main Authors: Handayani, Anik Nur, Amaliya, Sholikhatul, Akbar, Muhammad Iqbal, Wiryawan, Muhammad Zaki, Liang, Yeoh Wen, Kurniawan, Wendy Cahya
Format: Journal Article
Language:English
Published: 22.02.2025
ISSN:2775-2658, 2775-2658
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract SIBI is less widely adopted, and the lack of an efficient recognition system limits its accessibility. SIBI gestures often involve subtle hand movements and complex finger configurations, requiring precise feature extraction and classification techniques. This study addresses these issues using a Hand Keypoint-based Convolutional Neural Network (HK-CNN) for SIBI classification. The research utilizes Kinect 2.0 for precise data collection, enabling accurate hand keypoint detection and preprocessing. The optimal data acquisition distance between 50 and 60 cm from the camera is considered to obtain clear and detailed images. The methodology includes four key stages: data collection, preprocessing (keypoint extraction and image filtering), classification using HK-CNN with ResNet-50, EfficientNet, and InceptionV3, and performance evaluation. Experimental results demonstrate that EfficientNet achieves the highest accuracy of 99.1% in the 60:40 data split scenario, with superior precision and recall, making it ideal for real-time applications. ResNet-50 also performs well with 99.3% accuracy in the 20:80 split but requires longer computation time, while InceptionV3 is less efficient for real-time applications. Compared to traditional CNN methods, HK-CNN significantly enhances accuracy and efficiency. In conclusion, this study provides a robust and adaptable solution for SIBI recognition, facilitating inclusivity in education, public services, and workplace communication. Future research should expand dataset diversity and explore dynamic gesture recognition for further improvements.
AbstractList SIBI is less widely adopted, and the lack of an efficient recognition system limits its accessibility. SIBI gestures often involve subtle hand movements and complex finger configurations, requiring precise feature extraction and classification techniques. This study addresses these issues using a Hand Keypoint-based Convolutional Neural Network (HK-CNN) for SIBI classification. The research utilizes Kinect 2.0 for precise data collection, enabling accurate hand keypoint detection and preprocessing. The optimal data acquisition distance between 50 and 60 cm from the camera is considered to obtain clear and detailed images. The methodology includes four key stages: data collection, preprocessing (keypoint extraction and image filtering), classification using HK-CNN with ResNet-50, EfficientNet, and InceptionV3, and performance evaluation. Experimental results demonstrate that EfficientNet achieves the highest accuracy of 99.1% in the 60:40 data split scenario, with superior precision and recall, making it ideal for real-time applications. ResNet-50 also performs well with 99.3% accuracy in the 20:80 split but requires longer computation time, while InceptionV3 is less efficient for real-time applications. Compared to traditional CNN methods, HK-CNN significantly enhances accuracy and efficiency. In conclusion, this study provides a robust and adaptable solution for SIBI recognition, facilitating inclusivity in education, public services, and workplace communication. Future research should expand dataset diversity and explore dynamic gesture recognition for further improvements.
Author Wiryawan, Muhammad Zaki
Handayani, Anik Nur
Amaliya, Sholikhatul
Akbar, Muhammad Iqbal
Liang, Yeoh Wen
Kurniawan, Wendy Cahya
Author_xml – sequence: 1
  givenname: Anik Nur
  surname: Handayani
  fullname: Handayani, Anik Nur
– sequence: 2
  givenname: Sholikhatul
  surname: Amaliya
  fullname: Amaliya, Sholikhatul
– sequence: 3
  givenname: Muhammad Iqbal
  surname: Akbar
  fullname: Akbar, Muhammad Iqbal
– sequence: 4
  givenname: Muhammad Zaki
  orcidid: 0009-0005-9536-2657
  surname: Wiryawan
  fullname: Wiryawan, Muhammad Zaki
– sequence: 5
  givenname: Yeoh Wen
  surname: Liang
  fullname: Liang, Yeoh Wen
– sequence: 6
  givenname: Wendy Cahya
  surname: Kurniawan
  fullname: Kurniawan, Wendy Cahya
BookMark eNpNz0FLwzAYxvEgE5xzd4_5Aq1p0rdZvbmhrlgmOD2HNHlTIpqMZAr79rLpwdPzPz3wuySTEAMScl2xUlSyETf-PZlcfoPnZSVrOCNTLiUUvIHF5F9fkHnOfmB1LesKaj4lt2sdLH3Cwy76sC-WOqOlq82Gupjotlt2dOvHQHsdxi89In1BE8fg9z6GK3Lu9EfG-d_OyNvD_etqXfTPj93qri9MBQCF040AI6XjwqKQDIxlHAaurWatQ9Zi2yDnKITR1kJtLddsWMAgeOt4i2JG2O-vSTHnhE7tkv_U6aAqpk58deKrI18d-eIH_wBQaQ
ContentType Journal Article
DBID AAYXX
CITATION
DOI 10.31763/ijrcs.v5i2.1745
DatabaseName CrossRef
DatabaseTitle CrossRef
DatabaseTitleList CrossRef
DeliveryMethod fulltext_linktorsrc
EISSN 2775-2658
EndPage 829
ExternalDocumentID 10_31763_ijrcs_v5i2_1745
GroupedDBID AAYXX
ALMA_UNASSIGNED_HOLDINGS
ARCSS
CITATION
ID FETCH-LOGICAL-c1555-fa635c77f23de3705cd025b2ada09fe09e96e22e33cadd54dd2a0b85b329f29e3
ISSN 2775-2658
IngestDate Sat Nov 29 07:52:57 EST 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 2
Language English
License https://creativecommons.org/licenses/by-sa/4.0
LinkModel OpenURL
MergedId FETCHMERGED-LOGICAL-c1555-fa635c77f23de3705cd025b2ada09fe09e96e22e33cadd54dd2a0b85b329f29e3
ORCID 0009-0005-9536-2657
OpenAccessLink https://doi.org/10.31763/ijrcs.v5i2.1745
PageCount 17
ParticipantIDs crossref_primary_10_31763_ijrcs_v5i2_1745
PublicationCentury 2000
PublicationDate 2025-02-22
PublicationDateYYYYMMDD 2025-02-22
PublicationDate_xml – month: 02
  year: 2025
  text: 2025-02-22
  day: 22
PublicationDecade 2020
PublicationTitle International Journal of Robotics and Control Systems
PublicationYear 2025
SSID ssib044741542
ssib050928061
Score 2.2904398
Snippet SIBI is less widely adopted, and the lack of an efficient recognition system limits its accessibility. SIBI gestures often involve subtle hand movements and...
SourceID crossref
SourceType Index Database
StartPage 813
Title Hand Keypoint-Based CNN for SIBI Sign Language Recognition
Volume 5
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVHPJ
  databaseName: ROAD: Directory of Open Access Scholarly Resources
  customDbUrl:
  eissn: 2775-2658
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssib044741542
  issn: 2775-2658
  databaseCode: M~E
  dateStart: 20210101
  isFulltext: true
  titleUrlDefault: https://road.issn.org
  providerName: ISSN International Centre
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3Pb9MwFLbK4MBl2gQIBkM-7IKqDNdJ6phbVw2tglWIDjFxifxTy0rTruvKeuHf4t_jOU7SrAKJHbhEkRs9Jf0--X1-fu8ZoQPJCZHUhoGlVgWRTDqBUEIFiSSMJ5rIDtHFYRNsOEzOz_mnVutXVQuz_M7yPLm95bP_CjWMAdiudPYecNdGYQDuAXS4Auxw_SfgT1wo_INZzaZZvgiOwEvpdn84LPIJR4OjQXvkjuL8WMYpnW70GUQlPpfr1Pa7kcJStn6eymnd2rlfJro3-54X05krL_GnRbV7eTZ2QYmaWhOQ_qtCs45g6s3GF2Kxzk7sjaVP-T69uRCTidDtwZUU9c9fs_lK_PBB2_qJb6CCm9EL6qvBGwFNypjLsvPt2w_NH8bKWTpukJE2ZtzEl7JWztuHTzb9AoikrmtQkV3O1fXhMs6oS-SK1z6w2vffcI11wiIslQobaWEhdRZSZ-EBekhZzF0u4enP42oiiyKn06JaN4IocxvYbvFff5vfMi-Mvt14rYZEamidsx20XaKNe55cu6hl8ifonUMV3yUWBmJhIBZ2xMKOWLgiFm4Q6yn68v74rH8SlCdvBAr0ZRxYATpUMWZpqE3ISKw0YCep0IJwawg3vGsoNWGowD_GkdZUEJnEMqTcUm7CZ2grn-bmOcLWSilYZEJjQDDB4paLjlasoxJhQ9M1L9Cb6mPTmW-wkv7t3967x7Mv0eM1316hrcX8xuyjR2q5yK7nrwu4fgN9FHYK
linkProvider ISSN International Centre
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Hand+Keypoint-Based+CNN+for+SIBI+Sign+Language+Recognition&rft.jtitle=International+Journal+of+Robotics+and+Control+Systems&rft.au=Handayani%2C+Anik+Nur&rft.au=Amaliya%2C+Sholikhatul&rft.au=Akbar%2C+Muhammad+Iqbal&rft.au=Wiryawan%2C+Muhammad+Zaki&rft.date=2025-02-22&rft.issn=2775-2658&rft.eissn=2775-2658&rft.volume=5&rft.issue=2&rft.spage=813&rft.epage=829&rft_id=info:doi/10.31763%2Fijrcs.v5i2.1745&rft.externalDBID=n%2Fa&rft.externalDocID=10_31763_ijrcs_v5i2_1745
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2775-2658&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2775-2658&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2775-2658&client=summon