Think Fast: A Tensor Streaming Processor (TSP) for Accelerating Deep Learning Workloads

In this paper, we introduce the Tensor Streaming Processor (TSP) architecture, a functionally-sliced microarchitecture with memory units interleaved with vector and matrix deep learning functional units in order to take advantage of dataflow locality of deep learning operations. The TSP is built bas...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) s. 145 - 158
Hlavní autoři: Abts, Dennis, Ross, Jonathan, Sparling, Jonathan, Wong-VanHaren, Mark, Baker, Max, Hawkins, Tom, Bell, Andrew, Thompson, John, Kahsai, Temesghen, Kimmell, Garrin, Hwang, Jennifer, Leslie-Hurd, Rebekah, Bye, Michael, Creswick, E.R., Boyd, Matthew, Venigalla, Mahitha, Laforge, Evan, Purdy, Jon, Kamath, Purushotham, Maheshwari, Dinesh, Beidler, Michael, Rosseel, Geert, Ahmad, Omar, Gagarin, Gleb, Czekalski, Richard, Rane, Ashay, Parmar, Sahil, Werner, Jeff, Sproch, Jim, Macias, Adrian, Kurtz, Brian
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.05.2020
Témata:
Computer architecture
Data models
Deep learning
Hardware
Microarchitecture
Parallel processing
System-on-chip
Tensors
Transistors
Vectors
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In this paper, we introduce the Tensor Streaming Processor (TSP) architecture, a functionally-sliced microarchitecture with memory units interleaved with vector and matrix deep learning functional units in order to take advantage of dataflow locality of deep learning operations. The TSP is built based on two key observations: (1) machine learning workloads exhibit abundant data parallelism, which can be readily mapped to tensors in hardware, and (2) a simple and deterministic processor with producer-consumer stream programming model enables precise reasoning and control of hardware components, achieving good performance and power efficiency. The TSP is designed to exploit parallelism inherent in machine-learning workloads including instruction-level, memory concurrency, data and model parallelism, while guaranteeing determinism by eliminating all reactive elements in the hardware (e.g. arbiters, and caches). Early ResNet50 image classification results demonstrate 20.4K processed images per second (IPS) with a batch-size of one- a 4 \times improvement compared to other modern GPUs and accelerators [44]. Our first ASIC implementation of the TSP architecture yields a computational density of more than 1 TeraOp/s per square mm of silicon for its 25 \times 29 mm 14nm chip operating at a nominal clock frequency of 900 MHz. The TSP demonstrates a novel hardware-software approach to achieve fast, yet predictable, performance on machine-learning workloads within a desired power envelope.
DOI:10.1109/ISCA45697.2020.00023