DeftNN addressing bottlenecks for DNN execution on GPUs via synapse vector elimination and near-compute data fission

Deep neural networks (DNNs) are key computational building blocks for emerging classes of web services that interact in real time with users via voice, images and video inputs. Although GPUs have gained popularity as a key accelerator platform for deep learning workloads, the increasing demand for D...

Full description

Saved in:
Bibliographic Details
Published in:MICRO-50 : the 50th annual IEEE/ACM International Symposium on Microarchitecture : proceedings : October 14-18, 2017, Cambridge, MA pp. 786 - 799
Main Authors: Hill, Parker, Jain, Animesh, Hill, Mason, Zamirai, Babak, Hsu, Chang-Hong, Laurenzano, Michael A., Mahlke, Scott, Tang, Lingjia, Mars, Jason
Format: Conference Proceeding
Language:English
Published: New York, NY, USA ACM 14.10.2017
Series:ACM Conferences
Subjects:
Bandwidth
Computer systems organization > Architectures > Parallel architectures > Single instruction, multiple data
Computing methodologies > Machine learning > Machine learning approaches > Neural networks
Deep Neural Networks
General and reference > Cross-computing tools and techniques > Performance
GPU Architecture
Graphics processing units
Hardware
Memory Bandwidth
Memory management
Optimization
Performance Optimization
Synapses
System-on-chip
memory bandwidth
GPU architecture
performance optimization
deep neural networks
ISBN:1450349528, 9781450349529
ISSN:2379-3155
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Deep neural networks (DNNs) are key computational building blocks for emerging classes of web services that interact in real time with users via voice, images and video inputs. Although GPUs have gained popularity as a key accelerator platform for deep learning workloads, the increasing demand for DNN computation leaves a significant gap between the compute capabilities of GPU-enabled datacenters and the compute needed to service demand. The state-of-the-art techniques to improve DNN performance have significant limitations in bridging the gap on real systems. Current network pruning techniques remove computation, but the resulting networks map poorly to GPU architectures, yielding no performance benefit or even slowdowns. Meanwhile, current bandwidth optimization techniques focus on reducing off-chip bandwidth while overlooking on-chip bandwidth, a key DNN bottleneck. To address these limitations, this work introduces DeftNN, a GPU DNN execution framework that targets the key architectural bottlenecks of DNNs on GPUs to automatically and transparently improve execution performance. DeftNN is composed of two novel optimization techniques - (1) synapse vector elimination, a technique that identifies non-contributing synapses in the DNN and carefully transforms data and removes the computation and data movement of these synapses while fully utilizing the GPU to improve performance, and (2) near-compute data fission, a mechanism for scaling down the on-chip data movement requirements within DNN computations. Our evaluation of DeftNN spans 6 state-of-the-art DNNs. By applying both optimizations in concert, DeftNN is able to achieve an average speedup of 2.1X on real GPU hardware. We also introduce a small additional hardware unit per GPU core to facilitate efficient data fission operations, increasing the speedup achieved by DeftNN to 2.6X.
ISBN:1450349528
9781450349529
ISSN:2379-3155
DOI:10.1145/3123939.3123970