Unsupervised RGB-T object tracking with attentional multi-modal feature fusion

RGB-T tracking means that given the object position in the first frame, the tracker is trained to predict the position of the object in consecutive frames by taking full advantage of the complementary information of RGB and thermal infrared images. As the amount of data increases, unsupervised train...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Multimedia tools and applications Jg. 82; H. 15; S. 23595 - 23613
Hauptverfasser: Li, Shenglan, Yao, Rui, Zhou, Yong, Zhu, Hancheng, Liu, Bing, Zhao, Jiaqi, Shao, Zhiwen
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York Springer US 01.06.2023
Springer Nature B.V
Schlagworte:
Algorithms
Computer Communication Networks
Computer Science
Data Structures and Information Theory
Datasets
Deep learning
Feature extraction
Infrared imagery
Multimedia
Multimedia Information Systems
Neural networks
Optical tracking
Special Purpose and Application-Based Systems
Training
RGB-T object tracking
Attention mechanism
Unsupervised learning
ISSN:1380-7501, 1573-7721
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:RGB-T tracking means that given the object position in the first frame, the tracker is trained to predict the position of the object in consecutive frames by taking full advantage of the complementary information of RGB and thermal infrared images. As the amount of data increases, unsupervised training has great potential for development in RGB-T tracking task. As we all know, features extracted from different convolutional layers can provide different levels information in the image. In this paper, we propose a framework for visual tracking based on the attention mechanism fusion of multi-modal and multi-level features. This fusion method can give full play to the advantages of multi-level and multi-modal information. Specificly, we use a feature fusion module to fuse these features from different levels and different modalities at the same time. We use cycle consistency based on a correlation filter to implement unsupervised training of the model to reduce the cost of annotated data. The proposed tracker is evaluated on two popular benchmark datasets, GTOT and RGB-T234. Experimental results show that our tracker performs favorably against other state-of-the-art unsupervised trackers with a real-time tracking speed.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1380-7501
1573-7721
DOI:10.1007/s11042-023-14362-9