SRDD: a lightweight end-to-end object detection with transformer

Computer vision is now playing a vital role in modern UAV (Unmanned Aerial Vehicle) systems. However, the on-board real-time small object detection for UAVs remains challenging. This paper presents an end-to-end ViT (Vision Transformer) detector, named Sparse ROI-based Deformable DETR (SRDD), to mak...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Connection science Jg. 34; H. 1; S. 2448 - 2465
Hauptverfasser: Zhu, Yuan, Xia, Qingyuan, Jin, Wen
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Abingdon Taylor & Francis 31.12.2022
Taylor & Francis Ltd
Taylor & Francis Group
Schlagworte:
Coders
Computer vision
Deep learning
Deformation
Encoders-Decoders
Formability
Lightweight
lightweight neural architecture
object detection
Object recognition
Onboard equipment
Unmanned aerial vehicles
vision transformer
ISSN:0954-0091, 1360-0494
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Computer vision is now playing a vital role in modern UAV (Unmanned Aerial Vehicle) systems. However, the on-board real-time small object detection for UAVs remains challenging. This paper presents an end-to-end ViT (Vision Transformer) detector, named Sparse ROI-based Deformable DETR (SRDD), to make ViT model available to UAV on-board systems. We embed a scoring network in the transformer T-encoder to selectively prune the redundant tokens, at the same time, introduce ROI-based detection refinement module in the decoder to optimise detection performance while maintaining end-to-end detection pipeline. By using scoring networks, we compress the Transformer encoder/decoder to 1/3-layer structure, which is far slim compared with DETR. With the help of lightweight backbone ResT and dynamic anchor box, we relieve the memory insufficient of on-board SoC. Experiment on UAVDT dataset shows the proposed SRDD method achieved 50.2% mAP (outperforms Deformable DETR at least 7%). In addition, the lightweight version of SRDD achieved 51.08% mAP with 44% Params reduction.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0954-0091
1360-0494
DOI:10.1080/09540091.2022.2125499