Regional-CNN-based enhanced Turkish sign language recognition

Hearing impaired people use sign language for communication. Considering the spoken languages, the most widely spread ones have their own sign languages. Thus, sign languages are spoken language dependent. In this paper, the focus is on Turkish Sign Language (TSL). In parallel with the developments...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Signal, image and video processing Ročník 16; číslo 5; s. 1305 - 1311
Hlavní autori: Yirtici, Tolga, Yurtkan, Kamil
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: London Springer London 01.07.2022
Springer Nature B.V
Predmet:
Algorithms
Artificial intelligence
Artificial neural networks
Character recognition
Computer Imaging
Computer Science
Image Processing and Computer Vision
Languages
Machine learning
Multimedia Information Systems
Neural networks
Object recognition
Original Paper
Pattern Recognition and Graphics
Sign language
Signal,Image and Speech Processing
Vision
Sign language
Region based convolutional neural network
Turkish sign language
Transfer learning
Object detection
ISSN:1863-1703, 1863-1711
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Hearing impaired people use sign language for communication. Considering the spoken languages, the most widely spread ones have their own sign languages. Thus, sign languages are spoken language dependent. In this paper, the focus is on Turkish Sign Language (TSL). In parallel with the developments in computer science and artificial intelligence, the human computer interaction is now possible and it is a definite field of computer science. Automatically detecting and recognizing sign languages by computers are also possible applications. The improvements in machine learning field in the last decade made it possible to improve systems that can automatically detect and recognize sign language from still images. On the other hand, using a robust algorithm that is recognizing a sign language may not be applicable to another language as signs and linguistic properties may differ between languages. In this paper, we propose a novel method of recognizing characters of Turkish Sign Language (TSL). The method is tested on captured images containing signs, which are extracted from video files. Alexnet is employed as a pre-trained network in this system. Region-based Convolutional Neural Network (R-CNN) object detector is used to train the new model. The knowledge of the neural network is transferred using transfer learning method and tuned for recognizing TSL. The system achieves 99.7% average precision that is acceptable and comparable to conventional methods applied on TSL. Furthermore, the study is significant since it is one of the first successful application of transfer learning approach to TSL. The system is open to further improvements by improving the representations of the sign images.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1863-1703
1863-1711
DOI:10.1007/s11760-021-02082-2