Visual attention network

While originally designed for natural language processing tasks, the self-attention mechanism has recently taken various computer vision areas by storm. However, the 2D nature of images brings three challenges for applying self-attention in computer vision: (1) treating images as 1D sequences neglec...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Computational visual media (Beijing) Ročník 9; číslo 4; s. 733 - 752
Hlavní autoři: Guo, Meng-Hao, Lu, Cheng-Ze, Liu, Zheng-Ning, Cheng, Ming-Ming, Hu, Shi-Min
Médium: Journal Article
Jazyk:angličtina
Vydáno: Beijing Tsinghua University Press 01.12.2023
Springer Nature B.V
SpringerOpen
Témata:
Artificial Intelligence
Artificial neural networks
attention
Benchmarks
Computer Graphics
Computer Science
Computer vision
ConvNets
deep learning
Image classification
Image Processing and Computer Vision
Image resolution
Image segmentation
Natural language processing
Neural networks
Object recognition
Pose estimation
Research Article
Semantic segmentation
Semantics
User Interfaces and Human Computer Interaction
vision backbone
deep learning
ConvNets
vision backbone
attention
ISSN:2096-0433, 2096-0662
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:While originally designed for natural language processing tasks, the self-attention mechanism has recently taken various computer vision areas by storm. However, the 2D nature of images brings three challenges for applying self-attention in computer vision: (1) treating images as 1D sequences neglects their 2D structures; (2) the quadratic complexity is too expensive for high-resolution images; (3) it only captures spatial adaptability but ignores channel adaptability. In this paper, we propose a novel linear attention named large kernel attention (LKA) to enable self-adaptive and long-range correlations in self-attention while avoiding its shortcomings. Furthermore, we present a neural network based on LKA, namely Visual Attention Network (VAN). While extremely simple, VAN achieves comparable results with similar size convolutional neural networks (CNNs) and vision transformers (ViTs) in various tasks, including image classification, object detection, semantic segmentation, panoptic segmentation, pose estimation, etc. For example, VAN-B6 achieves 87.8% accuracy on ImageNet benchmark, and sets new state-of-the-art performance (58.2 PQ) for panoptic segmentation. Besides, VAN-B2 surpasses Swin-T 4 mIoU (50.1 vs. 46.1) for semantic segmentation on ADE20K benchmark, 2.6 AP (48.8 vs. 46.2) for object detection on COCO dataset. It provides a novel method and a simple yet strong baseline for the community. The code is available at https://github.com/Visual-Attention-Network .
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2096-0433
2096-0662
DOI:10.1007/s41095-023-0364-2