Dynamic energy-accuracy trade-off using stochastic computing in deep neural networks

This paper presents an efficient DNN design with stochastic computing. Observing that directly adopting stochastic computing to DNN has some challenges including random error fluctuation, range limitation, and overhead in accumulation, we address these problems by removing near-zero weights, applyin...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:2016 53nd ACM/EDAC/IEEE Design Automation Conference (DAC) S. 1 - 6
Hauptverfasser: Kyounghoon Kim, Jungki Kim, Joonsang Yu, Jungwoo Seo, Jongeun Lee, Kiyoung Choi
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 05.06.2016
Schlagworte:
Deep Learning
Deep Neural Networks
Encoding
Energy Efficiency
Hardware
Logic gates
Neurons
Signal to noise ratio
Stochastic Computing
Testing
Training
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This paper presents an efficient DNN design with stochastic computing. Observing that directly adopting stochastic computing to DNN has some challenges including random error fluctuation, range limitation, and overhead in accumulation, we address these problems by removing near-zero weights, applying weight-scaling, and integrating the activation function with the accumulator. The approach allows an easy implementation of early decision termination with a fixed hardware design by exploiting the progressive precision characteristics of stochastic computing, which was not easy with existing approaches. Experimental results show that our approach outperforms the conventional binary logic in terms of gate area, latency, and power consumption.
DOI:10.1145/2897937.2898011