EAML: ensemble self-attention-based mutual learning network for document image classification

In the recent past, complex deep neural networks have received huge interest in various document understanding tasks such as document image classification and document retrieval. As many document types have a distinct visual style, learning only visual features with deep CNNs to classify document im...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:International journal on document analysis and recognition Ročník 24; číslo 3; s. 251 - 268
Hlavní autoři: Bakkali, Souhail, Ming, Zuheng, Coustaty, Mickaël, Rusiñol, Marçal
Médium: Journal Article
Jazyk:angličtina
Vydáno: Berlin/Heidelberg Springer Berlin Heidelberg 01.09.2021
Springer Nature B.V
Témata:
Artificial neural networks
Classification
Computer Science
Image classification
Image Processing and Computer Vision
Machine learning
Modules
Pattern Recognition
Regularization
Special Issue Paper
Tobacco
Training
Self-attention-based fusion
Mutual learning
Multi-modal fusion
Ensemble learning
Text document image classification
ISSN:1433-2833, 1433-2825
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In the recent past, complex deep neural networks have received huge interest in various document understanding tasks such as document image classification and document retrieval. As many document types have a distinct visual style, learning only visual features with deep CNNs to classify document images has encountered the problem of low inter-class discrimination, and high intra-class structural variations between its categories. In parallel, text-level understanding jointly learned with the corresponding visual properties within a given document image has considerably improved the classification performance in terms of accuracy. In this paper, we design a self-attention-based fusion module that serves as a block in our ensemble trainable network. It allows to simultaneously learn the discriminant features of image and text modalities throughout the training stage. Besides, we encourage mutual learning by transferring the positive knowledge between image and text modalities during the training stage. This constraint is realized by adding a truncated Kullback–Leibler divergence loss (Tr- KLD Reg ) as a new regularization term, to the conventional supervised setting. To the best of our knowledge, this is the first time to leverage a mutual learning approach along with a self-attention-based fusion module to perform document image classification. The experimental results illustrate the effectiveness of our approach in terms of accuracy for the single-modal and multi-modal modalities. Thus, the proposed ensemble self-attention-based mutual learning model outperforms the state-of-the-art classification results based on the benchmark RVL-CDIP and Tobacco-3482 datasets.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1433-2833
1433-2825
DOI:10.1007/s10032-021-00378-0