SRDD: a lightweight end-to-end object detection with transformer

Computer vision is now playing a vital role in modern UAV (Unmanned Aerial Vehicle) systems. However, the on-board real-time small object detection for UAVs remains challenging. This paper presents an end-to-end ViT (Vision Transformer) detector, named Sparse ROI-based Deformable DETR (SRDD), to mak...

Full description

Saved in:
Bibliographic Details
Published in:Connection science Vol. 34; no. 1; pp. 2448 - 2465
Main Authors: Zhu, Yuan, Xia, Qingyuan, Jin, Wen
Format: Journal Article
Language:English
Published: Abingdon Taylor & Francis 31.12.2022
Taylor & Francis Ltd
Taylor & Francis Group
Subjects:
Coders
Computer vision
Deep learning
Deformation
Encoders-Decoders
Formability
Lightweight
lightweight neural architecture
object detection
Object recognition
Onboard equipment
Unmanned aerial vehicles
vision transformer
ISSN:0954-0091, 1360-0494
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Computer vision is now playing a vital role in modern UAV (Unmanned Aerial Vehicle) systems. However, the on-board real-time small object detection for UAVs remains challenging. This paper presents an end-to-end ViT (Vision Transformer) detector, named Sparse ROI-based Deformable DETR (SRDD), to make ViT model available to UAV on-board systems. We embed a scoring network in the transformer T-encoder to selectively prune the redundant tokens, at the same time, introduce ROI-based detection refinement module in the decoder to optimise detection performance while maintaining end-to-end detection pipeline. By using scoring networks, we compress the Transformer encoder/decoder to 1/3-layer structure, which is far slim compared with DETR. With the help of lightweight backbone ResT and dynamic anchor box, we relieve the memory insufficient of on-board SoC. Experiment on UAVDT dataset shows the proposed SRDD method achieved 50.2% mAP (outperforms Deformable DETR at least 7%). In addition, the lightweight version of SRDD achieved 51.08% mAP with 44% Params reduction.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0954-0091
1360-0494
DOI:10.1080/09540091.2022.2125499