Device Variation Effects on Neural Network Inference Accuracy in Analog In‐Memory Computing Systems

In analog in‐memory computing systems based on nonvolatile memories such as resistive random‐access memory (RRAM), neural network models are often trained offline and then the weights are programmed onto memory devices as conductance values. The programmed weight values inevitably deviate from the t...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Advanced intelligent systems Jg. 4; H. 8
Hauptverfasser: Wang, Qiwen, Park, Yongmo, Lu, Wei D.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Weinheim John Wiley & Sons, Inc 01.08.2022
Wiley
Schlagworte:
Accuracy
analog computing
Analog to digital converters
Arrays
Circuits
Computation
Datasets
deep neural networks
emerging memory
Energy efficiency
in-memory computing
Inference
Internet of Things
Memory devices
Neural networks
process-in-memory
Programming
RRAM
Training
Workloads
ISSN:2640-4567, 2640-4567
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In analog in‐memory computing systems based on nonvolatile memories such as resistive random‐access memory (RRAM), neural network models are often trained offline and then the weights are programmed onto memory devices as conductance values. The programmed weight values inevitably deviate from the target values during the programming process. This effect can be pronounced for emerging memories such as RRAM, PcRAM, and MRAM due to the stochastic nature during programming. Unlike noise, these weight deviations do not change during inference. The performance of neural network models is investigated against this programming variation under realistic system limitations, including limited device on/off ratios, memory array size, analog‐to‐digital converter (ADC) characteristics, and signed weight representations. Approaches to mitigate such device and circuit nonidealities through architecture‐aware training are also evaluated. The effectiveness of variation injection during training to improve the inference robustness, as well as the effects of different neural network training parameters such as learning rate schedule, will be discussed. In nonvolatile memory‐based analog in‐memory computing systems, variations during device programming can cause neural‐network inference accuracy to degrade since the stored weights will differ from those in the original models. Herein, the performance of deep neural‐network models is investigated against this effect under realistic system limitations, including limited device on/off ratios, memory array size, circuit characteristics, and signed weight representations.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2640-4567
2640-4567
DOI:10.1002/aisy.202100199