Student’s t-uniform mixture-based robust sparse coding model for sign language recognition from thermal images
For optical vision-based systems in human–robot interaction (HRI), the recognition performance is largely affected by environmental calamities e.g., photometric hazards and spatio-structural disruptions. Accounting for this vulnerability of optical RGB cameras, we propose a far infrared thermal imag...
Saved in:
| Published in: | Measurement : journal of the International Measurement Confederation Vol. 246; p. 116619 |
|---|---|
| Main Authors: | , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
Elsevier Ltd
31.03.2025
|
| Subjects: | |
| ISSN: | 0263-2241 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | For optical vision-based systems in human–robot interaction (HRI), the recognition performance is largely affected by environmental calamities e.g., photometric hazards and spatio-structural disruptions. Accounting for this vulnerability of optical RGB cameras, we propose a far infrared thermal imaging-based system in this work. Sign language (SL) plays a crucial role in bridging the communication gaps for individuals with hearing or speech impairments, enabling more inclusive interactions. We have considered the American manual alphabet (AMA) library-based SL images here. For modeling the distribution of coding error residuals effectively, we propose a novel Student’s t-uniform mixture error distribution framework. The corresponding sparse optimization problem is solved by an iteratively reweighted robust minimization algorithm. The property of Student’s t-distribution of having a strong peak at zero and an elongated tail on both sides makes it robust against higher coding residuals. The inclusion of a uniform distribution offers a flexible framework to accommodate data outliers e.g., random, unstructured errors, which may not conform to any specific patterns. This combined distribution allows the proposed model to more effectively handle the mixed presence of inliers and unpredictable outliers in the data. To validate this, extensive case studies are conducted using the normal-conditioned and pixel-degraded thermal sign images under various environmental challenges, such as low-light conditions, block occlusion, partial object occlusion, and noise corruption. Additional experiments are carried out with two benchmark hand gesture datasets as well. Together, these studies demonstrate the proposed technique to be an effective and robust sparse coding model under such challenging scenarios.
[Display omitted]
•Proposed FIR thermal imaging-based robust hand sign recognition system.•Introduced novel Student’s t-distribution-based error representation model.•Developed a novel mixture error model combining Student’s t and uniform distribution.•Validated the model with real-world FIR thermal images, including occlusion and noise.•Enhanced robust recognition performance under challenging environmental conditions. |
|---|---|
| ISSN: | 0263-2241 |
| DOI: | 10.1016/j.measurement.2024.116619 |