Dynamic Sparse Attention for Scalable Transformer Acceleration

Transformers are the mainstream of NLP applications and are becoming increasingly popular in other domains such as Computer Vision. Despite the improvements in model quality, the enormous computation costs make Transformers difficult at deployment, especially when the sequence length is large in eme...

Full description

Bibliographic Details

Published in:	IEEE transactions on computers Vol. 71; no. 12; pp. 1 - 14
Main Authors:	Liu, Liu, Qu, Zheng, Chen, Zhaodong, Tu, Fengbin, Ding, Yufei, Xie, Yuan
Format:	Journal Article
Language:	English
Published:	New York IEEE 01.01.2022 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:	Complexity Complexity theory Computational modeling Computer vision Costs Hardware Model accuracy Redundancy Sparse matrices Transformers
ISSN:	0018-9340, 1557-9956
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!