Dynamic energy-accuracy trade-off using stochastic computing in deep neural networks

This paper presents an efficient DNN design with stochastic computing. Observing that directly adopting stochastic computing to DNN has some challenges including random error fluctuation, range limitation, and overhead in accumulation, we address these problems by removing near-zero weights, applyin...

Full description

Saved in:
Bibliographic Details
Published in:2016 53nd ACM/EDAC/IEEE Design Automation Conference (DAC) pp. 1 - 6
Main Authors: Kyounghoon Kim, Jungki Kim, Joonsang Yu, Jungwoo Seo, Jongeun Lee, Kiyoung Choi
Format: Conference Proceeding
Language:English
Published: IEEE 05.06.2016
Subjects:
Deep Learning
Deep Neural Networks
Encoding
Energy Efficiency
Hardware
Logic gates
Neurons
Signal to noise ratio
Stochastic Computing
Testing
Training
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper presents an efficient DNN design with stochastic computing. Observing that directly adopting stochastic computing to DNN has some challenges including random error fluctuation, range limitation, and overhead in accumulation, we address these problems by removing near-zero weights, applying weight-scaling, and integrating the activation function with the accumulator. The approach allows an easy implementation of early decision termination with a fixed hardware design by exploiting the progressive precision characteristics of stochastic computing, which was not easy with existing approaches. Experimental results show that our approach outperforms the conventional binary logic in terms of gate area, latency, and power consumption.
DOI:10.1145/2897937.2898011