Structure-Aware in-Air Handwritten Text Recognition with Graph-Guided Cross-Modality Translator

In-air handwriting as a new human-computer interaction way plays an important role in many virtual/mixed-reality applications. Existing methods for in-air handwritten text recognition (IAHTR) typically directly process handwriting trajectories with deep neural networks. However, those methods all si...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings of the ... IEEE International Conference on Acoustics, Speech and Signal Processing (1998) pp. 5020 - 5024
Main Authors: Chen, Yuyan, Zhao, Xing, Gan, Ji, Leng, Jiaxu, Zhang, Yan, Gao, Xinbo
Format: Conference Proceeding
Language:English
Published: IEEE 14.04.2024
Subjects:
Costs
Encoder-Decoder
Fuses
Handwriting recognition
Handwritten Text Recognition
Human computer interaction
In-Air Handwriting
Multi-Modality Fusion
Signal processing
Speech recognition
Text recognition
ISSN:2379-190X
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In-air handwriting as a new human-computer interaction way plays an important role in many virtual/mixed-reality applications. Existing methods for in-air handwritten text recognition (IAHTR) typically directly process handwriting trajectories with deep neural networks. However, those methods all simply learn discriminative patterns by modelling low-level relationships between adjacent points of trajectories, while completely ignoring the inherent geometric structures of characters. Instead, we propose a novel Graph-guided Cross-modality Translator for IAHTR, which further explicitly exploits the geometric structures of characters for guiding the decoding of trajectories via graph-guided cross-modality attention mechanism without introducing extra annotation costs. Experiments on benchmarks IAHEW-UCAS2016 & IAM-OnDB show that our method has achieved state-of-the-art performance for handwritten text recognition.
ISSN:2379-190X
DOI:10.1109/ICASSP48485.2024.10447531