ByteTransformer: A High-Performance Transformer Boosted for Variable-Length Inputs

Transformers have become keystone models in natural language processing over the past decade. They have achieved great popularity in deep learning applications, but the increasing sizes of the parameter spaces required by transformer models generate a commensurate need to accelerate performance. Nat...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	Proceedings - IEEE International Parallel and Distributed Processing Symposium S. 344 - 355
Hauptverfasser:	Zhai, Yujia, Jiang, Chengquan, Wang, Leyuan, Jia, Xiaoying, Zhang, Shang, Chen, Zizhong, Liu, Xin, Zhu, Yibo
Format:	Tagungsbericht
Sprache:	Englisch
Veröffentlicht:	IEEE 01.05.2023
Schlagworte:	BERT Bit error rate CUTLASS Deep learning Distributed processing Graphics processing units Large Language Models Multi-head Attention Natural Language Processing NVIDIA GPU Optimization methods Technological innovation Training Transformer
ISSN:	1530-2075
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!