NetBooster: Empowering Tiny Deep Learning By Standing on the Shoulders of Deep Giants

Tiny deep learning has attracted increasing attention driven by the substantial demand for deploying deep learning on numerous intelligent Internet-of-Things devices. However, it is still challenging to unleash tiny deep learning's full potential on both large-scale datasets and downstream task...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	2023 60th ACM/IEEE Design Automation Conference (DAC) S. 1 - 6
Hauptverfasser:	Yu, Zhongzhi, Fu, Yonggan, Yuan, Jiayi, You, Haoran, Lin, Yingyan Celine
Format:	Tagungsbericht
Sprache:	Englisch
Veröffentlicht:	IEEE 09.07.2023
Schlagworte:	Boosting Deep learning Design automation Efficient Deep Learning Neural networks Task analysis Tiny Neural Networks Training
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Beschreibung
Zusammenfassung:	Tiny deep learning has attracted increasing attention driven by the substantial demand for deploying deep learning on numerous intelligent Internet-of-Things devices. However, it is still challenging to unleash tiny deep learning's full potential on both large-scale datasets and downstream tasks due to the under-fitting issues caused by the limited model capacity of tiny neural networks (TNNs). To this end, we propose a framework called NetBooster to empower tiny deep learning by augmenting the architectures of TNNs via an expansion-then-contraction strategy. Extensive experiments show that NetBooster consistently outperforms state-of-the-art tiny deep learning solutions.
DOI:	10.1109/DAC56929.2023.10247827