EAML: ensemble self-attention-based mutual learning network for document image classification

In the recent past, complex deep neural networks have received huge interest in various document understanding tasks such as document image classification and document retrieval. As many document types have a distinct visual style, learning only visual features with deep CNNs to classify document im...

Full description

Saved in:
Bibliographic Details
Published in:International journal on document analysis and recognition Vol. 24; no. 3; pp. 251 - 268
Main Authors: Bakkali, Souhail, Ming, Zuheng, Coustaty, Mickaël, Rusiñol, Marçal
Format: Journal Article
Language:English
Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.09.2021
Springer Nature B.V
Subjects:
Artificial neural networks
Classification
Computer Science
Image classification
Image Processing and Computer Vision
Machine learning
Modules
Pattern Recognition
Regularization
Special Issue Paper
Tobacco
Training
Self-attention-based fusion
Mutual learning
Multi-modal fusion
Ensemble learning
Text document image classification
ISSN:1433-2833, 1433-2825
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In the recent past, complex deep neural networks have received huge interest in various document understanding tasks such as document image classification and document retrieval. As many document types have a distinct visual style, learning only visual features with deep CNNs to classify document images has encountered the problem of low inter-class discrimination, and high intra-class structural variations between its categories. In parallel, text-level understanding jointly learned with the corresponding visual properties within a given document image has considerably improved the classification performance in terms of accuracy. In this paper, we design a self-attention-based fusion module that serves as a block in our ensemble trainable network. It allows to simultaneously learn the discriminant features of image and text modalities throughout the training stage. Besides, we encourage mutual learning by transferring the positive knowledge between image and text modalities during the training stage. This constraint is realized by adding a truncated Kullback–Leibler divergence loss (Tr- KLD Reg ) as a new regularization term, to the conventional supervised setting. To the best of our knowledge, this is the first time to leverage a mutual learning approach along with a self-attention-based fusion module to perform document image classification. The experimental results illustrate the effectiveness of our approach in terms of accuracy for the single-modal and multi-modal modalities. Thus, the proposed ensemble self-attention-based mutual learning model outperforms the state-of-the-art classification results based on the benchmark RVL-CDIP and Tobacco-3482 datasets.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1433-2833
1433-2825
DOI:10.1007/s10032-021-00378-0