An automatic RTL compiler for high-throughput FPGA implementation of diverse deep convolutional neural networks
Convolutional neural networks (CNNs) are rapidly evolving and being applied to a broad range of applications. Given a specific application, an increasing challenge is to search the appropriate CNN algorithm and efficiently map it to the target hardware. The FPGA-based accelerator has the advantage o...
Uloženo v:
| Vydáno v: | International Conference on Field-programmable Logic and Applications s. 1 - 8 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
Ghent University
01.09.2017
|
| Témata: | |
| ISSN: | 1946-1488 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Convolutional neural networks (CNNs) are rapidly evolving and being applied to a broad range of applications. Given a specific application, an increasing challenge is to search the appropriate CNN algorithm and efficiently map it to the target hardware. The FPGA-based accelerator has the advantage of reconfigurability and flexibility, and has achieved high-performance and low-power. Without a general compiler to automate the implementation, however, significant efforts and expertise are still required to customize the design for each CNN model. In this work, we present an RTL-level CNN compiler that automatically generates customized FPGA hardware for the inference tasks of various CNNs, in order to enable high-level fast prototyping of CNNs from software to FPGA and still keep the benefits of low-level hardware optimization. First, a general-purpose library of RTL modules is developed to model different operations at each layer. The implementation of each module is optimized at the RTL level. Given a CNN algorithm, its structure is abstracted to a directed acyclic graph (DAG) and then complied with RTL modules in the library. The integration and dataflow of physical modules are predefined in the top-level system template and reconfigured during compilation. The runtime control of layer-by-layer sequential computation is managed by the proposed execution schedule so that even highly irregular and complex network topology, e.g. ResNet, can be compiled. The proposed methodology is demonstrated with end-to-end FPGA implementations of various CNN algorithms (e.g. NiN, VGG-16, ResNet-50, and ResNet-152) on two standalone Intel FPGAs, Stratix V and Arria 10. The performance and overhead of the automated compilation are evaluated. The compiled FPGA accelerators exhibit superior performance compared to state-of-the-art automation-based works by >2× for various CNNs. |
|---|---|
| AbstractList | Convolutional neural networks (CNNs) are rapidly evolving and being applied to a broad range of applications. Given a specific application, an increasing challenge is to search the appropriate CNN algorithm and efficiently map it to the target hardware. The FPGA-based accelerator has the advantage of reconfigurability and flexibility, and has achieved high-performance and low-power. Without a general compiler to automate the implementation, however, significant efforts and expertise are still required to customize the design for each CNN model. In this work, we present an RTL-level CNN compiler that automatically generates customized FPGA hardware for the inference tasks of various CNNs, in order to enable high-level fast prototyping of CNNs from software to FPGA and still keep the benefits of low-level hardware optimization. First, a general-purpose library of RTL modules is developed to model different operations at each layer. The implementation of each module is optimized at the RTL level. Given a CNN algorithm, its structure is abstracted to a directed acyclic graph (DAG) and then complied with RTL modules in the library. The integration and dataflow of physical modules are predefined in the top-level system template and reconfigured during compilation. The runtime control of layer-by-layer sequential computation is managed by the proposed execution schedule so that even highly irregular and complex network topology, e.g. ResNet, can be compiled. The proposed methodology is demonstrated with end-to-end FPGA implementations of various CNN algorithms (e.g. NiN, VGG-16, ResNet-50, and ResNet-152) on two standalone Intel FPGAs, Stratix V and Arria 10. The performance and overhead of the automated compilation are evaluated. The compiled FPGA accelerators exhibit superior performance compared to state-of-the-art automation-based works by >2× for various CNNs. |
| Author | Seo, Jae-sun Vrudhula, Sarma Cao, Yu Ma, Yufei |
| Author_xml | – sequence: 1 givenname: Yufei surname: Ma fullname: Ma, Yufei email: yufeima@asu.edu organization: School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, USA – sequence: 2 givenname: Yu surname: Cao fullname: Cao, Yu email: yu.cao@asu.edu organization: School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, USA – sequence: 3 givenname: Sarma surname: Vrudhula fullname: Vrudhula, Sarma email: vrudhula@asu.edu organization: School of Computing, Informatics, Decision Systems Engineering, Arizona State University, Tempe, USA – sequence: 4 givenname: Jae-sun surname: Seo fullname: Seo, Jae-sun email: jaesun.seo@asu.edu organization: School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, USA |
| BookMark | eNotUMlOwzAUNAgk2tIPQFz8Aynemvgdq4oWpEhUqJwrK31uDIkdOU4Rf09Y5jKHWTSaKbnywSMhd5wthAQOD5tduRCMFwvNlrkW6oJMgQGTTAktLsmEg8ozrrS-IfO-f2cjlqrQy3xCwspTM6TQmuQq-rovaRXazjUYqQ2R1u5UZ6mOYTjV3ZDoZrddUdd2Dbbo05gJngZLj-6MsUd6ROzGAn8OzfCjmYZ6HOIvpc8QP_pbcm1N0-P8n2fkbfO4Xz9l5cv2eb0qMycUTxmoqtJHwYS12gBwxSu0RiEyzcfpRhejwyIwA1yCViBzNGgkcl4YqUHOyP1fr0PEQxdda-LX4f8f-Q3ECF0L |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.23919/FPL.2017.8056824 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9090304282 9789090304281 |
| EISSN | 1946-1488 |
| EndPage | 8 |
| ExternalDocumentID | 8056824 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IF 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK OCL RIE RIL |
| ID | FETCH-LOGICAL-i241t-94cc8d202ff8a99141cefa4ee081547a8794cfe90a913984936eaea3e117a3893 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 68 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000426989400069&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 20 06:20:57 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i241t-94cc8d202ff8a99141cefa4ee081547a8794cfe90a913984936eaea3e117a3893 |
| PageCount | 8 |
| ParticipantIDs | ieee_primary_8056824 |
| PublicationCentury | 2000 |
| PublicationDate | 2017-09 |
| PublicationDateYYYYMMDD | 2017-09-01 |
| PublicationDate_xml | – month: 09 year: 2017 text: 2017-09 |
| PublicationDecade | 2010 |
| PublicationTitle | International Conference on Field-programmable Logic and Applications |
| PublicationTitleAbbrev | FPL |
| PublicationYear | 2017 |
| Publisher | Ghent University |
| Publisher_xml | – name: Ghent University |
| SSID | ssj0000547856 |
| Score | 2.044809 |
| Snippet | Convolutional neural networks (CNNs) are rapidly evolving and being applied to a broad range of applications. Given a specific application, an increasing... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 1 |
| SubjectTerms | Convolutional neural networks Field programmable gate arrays Hardware Libraries Optimization Runtime Schedules Software Software algorithms Topology |
| Title | An automatic RTL compiler for high-throughput FPGA implementation of diverse deep convolutional neural networks |
| URI | https://ieeexplore.ieee.org/document/8056824 |
| WOSCitedRecordID | wos000426989400069&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFH5s4sGTyib-JgePdmvarEmOQ6wexigyZbeRJq8w0HZsrX-_SVonghdPKSGl8F7S7_Hyve8B3MU5S5TIdcB1NAkYDY39D2IRUGXdj1xT45M5bzM-n4vlUmY9uN_XwiCiJ5_hyD36u3xT6calysbCorWIWB_6nPO2VmufTwmdMNUkaS8uo1hSOU6zmeNu8VH33q8GKh4_0uP_ffkEhj-FeCTbQ8wp9LAcQDUtiWrqyoutkpfFjDheuD3dW2IjUOIEiIOu_c6mqUmaPU3J-uObJ-4cQaqCGE_IQGIQN8Rxz7s9qN6J07j0g2eI74bwmj4uHp6Drm9CsLZ4XAeSaS1MFEZFIZSN_xjVWCiGaOHfWkkJewZ1gTJUThNUMBknqFDFSClXLoA5g4OyKvEcCKPSRiwmpzrSDKkWPJe5YaGxaw1L9AUMnLFWm1YaY9XZ6fLv6Ss4cv5oKVrXcFBvG7yBQ_1Zr3fbW-_PL9BIo_c |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA5zCnpS2cTf5uDRzibN2uQ4xDqxjiJTdhtp8goDbcfW-vebpHUiePGUElIK7yV9Hy_f-x5C10HGQskz5UWKDj1GfG3-g5B7RBr3Q6SIdsmctySaTPhsJtIOutnUwgCAI5_BwD66u3xdqtqmym65idacsi20PWSMkqZaa5NR8a001TBsri5pIIi4jdPEsreiQfvmrxYqLoLE-__79gHq_5Ti4XQTZA5RB4oeKkcFlnVVOrlV_DJNsGWGm_O9wgaDYitB7LUNeJZ1heP0YYQXH99McesKXOZYO0oGYA2wxJZ93u5C-Y6tyqUbHEd83Uev8f30buy1nRO8hYnIlSeYUlxTn-Y5lwYBMqIglwzAAABjJcnNKVQ5CF9aVVDORBCCBBkAIZG0EOYIdYuygGOEGREGs-iMKKoYEMWjTGSa-dqs1SxUJ6hnjTVfNuIY89ZOp39PX6Hd8fQ5mSePk6cztGd90xC2zlG3WtVwgXbUZ7VYry6db78AgsenPg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=International+Conference+on+Field-programmable+Logic+and+Applications&rft.atitle=An+automatic+RTL+compiler+for+high-throughput+FPGA+implementation+of+diverse+deep+convolutional+neural+networks&rft.au=Ma%2C+Yufei&rft.au=Cao%2C+Yu&rft.au=Vrudhula%2C+Sarma&rft.au=Seo%2C+Jae-sun&rft.date=2017-09-01&rft.pub=Ghent+University&rft.eissn=1946-1488&rft.spage=1&rft.epage=8&rft_id=info:doi/10.23919%2FFPL.2017.8056824&rft.externalDocID=8056824 |