An Efficient Training Accelerator for Transformers With Hardware-Algorithm Co-Optimization

Transformers have achieved significant success in deep learning, and training Transformers efficiently on resource-constrained platforms has been attracting continuous attention for domain adaptions and privacy concerns. However, deploying Transformers training on these platforms is still challengin...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE transactions on very large scale integration (VLSI) systems Vol. 31; no. 11; pp. 1788 - 1801
Main Authors:	Shao, Haikuo, Lu, Jinming, Wang, Meiqi, Wang, Zhongfeng
Format:	Journal Article
Language:	English
Published:	New York IEEE 01.11.2023 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:	Algorithm-hardware codesign Algorithms Computational modeling Computer architecture Computer memory Data models Energy efficiency general matrix multiplication (GEMM) Hardware Memory management nonlinear function Optimization Platforms Task analysis Technology assessment Training training accelerator Transformer Transformers
ISSN:	1063-8210, 1557-9999
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Be the first to leave a comment!