Hand Keypoint-Based CNN for SIBI Sign Language Recognition

SIBI is less widely adopted, and the lack of an efficient recognition system limits its accessibility. SIBI gestures often involve subtle hand movements and complex finger configurations, requiring precise feature extraction and classification techniques. This study addresses these issues using a Ha...

Full description

Saved in:
Bibliographic Details
Published in:International Journal of Robotics and Control Systems Vol. 5; no. 2; pp. 813 - 829
Main Authors: Handayani, Anik Nur, Amaliya, Sholikhatul, Akbar, Muhammad Iqbal, Wiryawan, Muhammad Zaki, Liang, Yeoh Wen, Kurniawan, Wendy Cahya
Format: Journal Article
Language:English
Published: 22.02.2025
ISSN:2775-2658, 2775-2658
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:SIBI is less widely adopted, and the lack of an efficient recognition system limits its accessibility. SIBI gestures often involve subtle hand movements and complex finger configurations, requiring precise feature extraction and classification techniques. This study addresses these issues using a Hand Keypoint-based Convolutional Neural Network (HK-CNN) for SIBI classification. The research utilizes Kinect 2.0 for precise data collection, enabling accurate hand keypoint detection and preprocessing. The optimal data acquisition distance between 50 and 60 cm from the camera is considered to obtain clear and detailed images. The methodology includes four key stages: data collection, preprocessing (keypoint extraction and image filtering), classification using HK-CNN with ResNet-50, EfficientNet, and InceptionV3, and performance evaluation. Experimental results demonstrate that EfficientNet achieves the highest accuracy of 99.1% in the 60:40 data split scenario, with superior precision and recall, making it ideal for real-time applications. ResNet-50 also performs well with 99.3% accuracy in the 20:80 split but requires longer computation time, while InceptionV3 is less efficient for real-time applications. Compared to traditional CNN methods, HK-CNN significantly enhances accuracy and efficiency. In conclusion, this study provides a robust and adaptable solution for SIBI recognition, facilitating inclusivity in education, public services, and workplace communication. Future research should expand dataset diversity and explore dynamic gesture recognition for further improvements.
ISSN:2775-2658
2775-2658
DOI:10.31763/ijrcs.v5i2.1745