Dynamic Sparse Attention for Scalable Transformer Acceleration

Transformers are the mainstream of NLP applications and are becoming increasingly popular in other domains such as Computer Vision. Despite the improvements in model quality, the enormous computation costs make Transformers difficult at deployment, especially when the sequence length is large in eme...

Celý popis

Podrobná bibliografia

Vydané v:	IEEE transactions on computers Ročník 71; číslo 12; s. 1 - 14
Hlavní autori:	Liu, Liu, Qu, Zheng, Chen, Zhaodong, Tu, Fengbin, Ding, Yufei, Xie, Yuan
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	New York IEEE 01.01.2022 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:	Complexity Complexity theory Computational modeling Computer vision Costs Hardware Model accuracy Redundancy Sparse matrices Transformers
ISSN:	0018-9340, 1557-9956
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!