Dynamic Sparse Attention for Scalable Transformer Acceleration

Transformers are the mainstream of NLP applications and are becoming increasingly popular in other domains such as Computer Vision. Despite the improvements in model quality, the enormous computation costs make Transformers difficult at deployment, especially when the sequence length is large in eme...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	IEEE transactions on computers Jg. 71; H. 12; S. 1 - 14
Hauptverfasser:	Liu, Liu, Qu, Zheng, Chen, Zhaodong, Tu, Fengbin, Ding, Yufei, Xie, Yuan
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	New York IEEE 01.01.2022 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Complexity Complexity theory Computational modeling Computer vision Costs Hardware Model accuracy Redundancy Sparse matrices Transformers
ISSN:	0018-9340, 1557-9956
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!