Bulk‐Switching Memristor‐Based Compute‐In‐Memory Module for Deep Neural Network Training

The constant drive to achieve higher performance in deep neural networks (DNNs) has led to the proliferation of very large models. Model training, however, requires intensive computation time and energy. Memristor‐based compute‐in‐memory (CIM) modules can perform vector‐matrix multiplication (VMM) i...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	Advanced materials (Weinheim) Ročník 35; číslo 46; s. e2305465 - n/a
Hlavní autoři:	Wu, Yuting, Wang, Qiwen, Wang, Ziyu, Wang, Xinxin, Ayyagari, Buvna, Krishnan, Siddarth, Chudzik, Michael, Lu, Wei D.
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	Weinheim Wiley Subscription Services, Inc 01.11.2023
Témata:	Artificial neural networks Computer memory deep neural network training deep neural networks Hardware Inference in‐memory computing Materials science Memristors mixed precision training Modules Neural networks Switching Training
ISSN:	0935-9648, 1521-4095, 1521-4095
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Popis
Shrnutí:	The constant drive to achieve higher performance in deep neural networks (DNNs) has led to the proliferation of very large models. Model training, however, requires intensive computation time and energy. Memristor‐based compute‐in‐memory (CIM) modules can perform vector‐matrix multiplication (VMM) in place and in parallel, and have shown great promises in DNN inference applications. However, CIM‐based model training faces challenges due to non‐linear weight updates, device variations, and low‐precision. In this work, a mixed‐precision training scheme is experimentally implemented to mitigate these effects using a bulk‐switching memristor‐based CIM module. Low‐precision CIM modules are used to accelerate the expensive VMM operations, with high‐precision weight updates accumulated in digital units. Memristor devices are only changed when the accumulated weight update value exceeds a pre‐defined threshold. The proposed scheme is implemented with a system‐onchip of fully integrated analog CIM modules and digital sub‐systems, showing fast convergence of LeNet training to 97.73%. The efficacy of training larger models is evaluated using realistic hardware parameters and verifies that CIM modules can enable efficient mix‐precision DNN training with accuracy comparable to full‐precision software‐trained models. Additionally, models trained on chip are inherently robust to hardware variations, allowing direct mapping to CIM inference chips without additional re‐training. A compute‐in‐memory module integrated with bulk‐switching memristor arrays is demonstrated. A hardware–software co‐designed system is constructed to accelerate the neural network training and relax stringent requirements for device precision and endurance. The on‐chip trained networks are able to reach software‐comparable accuracies and exhibit robustness to weight perturbation when transferred to a new chip.
Bibliografie:	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 content type line 23
ISSN:	0935-9648 1521-4095 1521-4095
DOI:	10.1002/adma.202305465