A CNN-LSTM based ensemble framework for in-air handwritten Assamese character recognition

In-air handwriting is a contemporary human computer interaction (HCI) technique which enables users to write and communicate in free space in a simple and intuitive manner. Air-written characters exhibit wide variations depending upon different writing styles of users and their speed of articulation...

Full description

Saved in:
Bibliographic Details
Published in:Multimedia tools and applications Vol. 80; no. 28-29; pp. 35649 - 35684
Main Authors: Choudhury, Ananya, Sarma, Kandarpa Kumar
Format: Journal Article
Language:English
Published: New York Springer US 01.11.2021
Springer Nature B.V
Subjects:
1166: Advances of machine learning in data analytics and visual information processing
Artificial neural networks
Character recognition
Computer Communication Networks
Computer Science
Data Structures and Information Theory
Feature recognition
Fields (mathematics)
Handwriting
Handwriting recognition
Modelling
Multimedia Information Systems
Neural networks
Segmentation
Segments
Special Purpose and Application-Based Systems
Writing
In-air handwriting recognition
Human computer interaction
Long short-term memory
Ligature
Markov random field
Convolutional neural network
ISSN:1380-7501, 1573-7721
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In-air handwriting is a contemporary human computer interaction (HCI) technique which enables users to write and communicate in free space in a simple and intuitive manner. Air-written characters exhibit wide variations depending upon different writing styles of users and their speed of articulation, which presents a great challenge towards effective recognition of linguistic characters. So, in this paper we have proposed an ensemble model for in-air handwriting recognition which is based on convolutional neural network (CNN) and a long short-term memory neural network (LSTM-NN). The method collaborates overall character trajectory appearance modeling and temporal trajectory feature modeling for efficient recognition of varied types of air-written characters. In contrast to two-dimensional handwriting, in-air handwriting generally involves writing of characters interlinked by a continuous stroke, which makes segregation of intended writing activity from insignificant connecting motions an intricate task. So, a two-stage statistical framework is incorporated in the system for automatic detection and extraction of relevant writing segments from air-written characters. Identification of writing events from a continuous stream of air-written data is accomplished by formulating a Markov Random Field (MRF) model, while the segmentation of writing events into meaningful handwriting segments and redundant parts is performed by implementation of a Mahalanobis distance (MD) classifier. The proposed approach is assessed on an air-written character dataset comprising of Assamese vowels, consonants and numerals. The experimental results connote that our hybrid network can assimilate more information from the air-writing patterns and hence offer better recognition performance than the state-of-the-art approaches.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1380-7501
1573-7721
DOI:10.1007/s11042-020-10470-y