LNS-Madam: Low-Precision Training in Logarithmic Number System Using Multiplicative Weight Update

Representing deep neural networks (DNNs) in low-precision is a promising approach to enable efficient acceleration and memory reduction. Previous methods that train DNNs in low-precision typically keep a copy of weights in high-precision during the weight updates. Directly training with low-precisio...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE transactions on computers Vol. 71; no. 12; pp. 3179 - 3190
Main Authors:	Zhao, Jiawei, Dai, Steve, Venkatesan, Rangharajan, Zimmer, Brian, Ali, Mustafa, Liu, Ming-Yu, Khailany, Brucek, Dally, William J., Anandkumar, Anima
Format:	Journal Article
Language:	English
Published:	New York IEEE 01.12.2022 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:	Algorithms Arithmetic Artificial neural networks Computer vision Costs Energy consumption Hardware hardware acceleration Logarithms Machine learning Natural language (computers) Neural networks Number systems optimization Quantization (signal) Table lookup Training
ISSN:	0018-9340, 1557-9956
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Be the first to leave a comment!