Linear Approximation of Deep Neural Networks for Efficient Inference on Video Data
Sequential data such as video are characterized by spatio-temporal correlations. As of yet, few deep learning algorithms exploit them to decrease the often massive cost during inference. This work leverages correlations in video data to linearize part of a deep neural network and thus reduce its siz...
Uloženo v:
| Vydáno v: | 2019 27th European Signal Processing Conference (EUSIPCO) s. 1 - 5 |
|---|---|
| Hlavní autoři: | , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
EURASIP
01.09.2019
|
| Témata: | |
| ISSN: | 2076-1465 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Sequential data such as video are characterized by spatio-temporal correlations. As of yet, few deep learning algorithms exploit them to decrease the often massive cost during inference. This work leverages correlations in video data to linearize part of a deep neural network and thus reduce its size and computational cost. Drawing upon the simplicity of the typically used rectifier activation function, we replace the ReLU function by dynamically updating masks. The resulting layer stack is a simple chain of matrix multiplications and bias additions, that can be contracted into a single weight matrix and bias vector. Inference then reduces to an affine transformation of the input sequence with these contracted parameters. We show that the method is akin to approximating the neural network with a first-order Taylor expansion around a dynamically updating reference point. The proposed algorithm is evaluated on a denoising convolutional autoencoder. |
|---|---|
| AbstractList | Sequential data such as video are characterized by spatio-temporal correlations. As of yet, few deep learning algorithms exploit them to decrease the often massive cost during inference. This work leverages correlations in video data to linearize part of a deep neural network and thus reduce its size and computational cost. Drawing upon the simplicity of the typically used rectifier activation function, we replace the ReLU function by dynamically updating masks. The resulting layer stack is a simple chain of matrix multiplications and bias additions, that can be contracted into a single weight matrix and bias vector. Inference then reduces to an affine transformation of the input sequence with these contracted parameters. We show that the method is akin to approximating the neural network with a first-order Taylor expansion around a dynamically updating reference point. The proposed algorithm is evaluated on a denoising convolutional autoencoder. |
| Author | Rueckauer, Bodo Liu, Shih-Chii |
| Author_xml | – sequence: 1 givenname: Bodo surname: Rueckauer fullname: Rueckauer, Bodo organization: Institute of Neuroinformatics, University of Zurich and ETH Zurich,Zurich,Switzerland – sequence: 2 givenname: Shih-Chii surname: Liu fullname: Liu, Shih-Chii organization: Institute of Neuroinformatics, University of Zurich and ETH Zurich,Zurich,Switzerland |
| BookMark | eNotkNtKxDAYhKMouK59AhHyAl2TP20Ol8setFBcUdfbJW3_QHBNSlpR396CezVzM8M3c00uQgxIyB1nCxCGm_vN_rV6Xu0WwLhZaMPAGHVGMqMmr0EZxYQ5JzNgSua8kOUVyYbBNww004ozOSMvtQ9oE132fYo__tOOPgYaHV0j9vQJv5I9TjJ-x_QxUBcT3TjnW49hpFVwmDC0SKfIu-8w0rUd7Q25dPY4YHbSOdlvN2-rx7zePVSrZZ17KNmYWyUZdly6UlgolcaiUNq6xk7oHEFw2QJ0jJfaAcpCThtQC0DRNKWUthFzcvvf6xHx0KcJPv0eTi-IP_yzUsM |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.23919/EUSIPCO.2019.8902997 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Xplore url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9789082797039 9082797038 |
| EISSN | 2076-1465 |
| EndPage | 5 |
| ExternalDocumentID | 8902997 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IL ACGFS ALMA_UNASSIGNED_HOLDINGS CBEJK RIE RIL |
| ID | FETCH-LOGICAL-i250t-a760ed16f53a2578e4478afba9081e2316c22d0158f2e646076e832e3bb566ab3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000604567700320&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:31:53 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i250t-a760ed16f53a2578e4478afba9081e2316c22d0158f2e646076e832e3bb566ab3 |
| PageCount | 5 |
| ParticipantIDs | ieee_primary_8902997 |
| PublicationCentury | 2000 |
| PublicationDate | 2019-Sept. |
| PublicationDateYYYYMMDD | 2019-09-01 |
| PublicationDate_xml | – month: 09 year: 2019 text: 2019-Sept. |
| PublicationDecade | 2010 |
| PublicationTitle | 2019 27th European Signal Processing Conference (EUSIPCO) |
| PublicationTitleAbbrev | EUSIPCO |
| PublicationYear | 2019 |
| Publisher | EURASIP |
| Publisher_xml | – name: EURASIP |
| SSID | ssib028087106 ssib025355106 |
| Score | 1.7050045 |
| Snippet | Sequential data such as video are characterized by spatio-temporal correlations. As of yet, few deep learning algorithms exploit them to decrease the often... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 1 |
| SubjectTerms | Biological neural networks compression Convolution Correlation Deep neural networks linearization Neurons Noise reduction sequential data Task analysis Taylor series video |
| Title | Linear Approximation of Deep Neural Networks for Efficient Inference on Video Data |
| URI | https://ieeexplore.ieee.org/document/8902997 |
| WOSCitedRecordID | wos000604567700320&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8NAEB3a4sGTSitqVfbg0dh87maP0g_spRa10lvZTSYQkKa0qfjznUlqVfDiKSGQTZjMMjOZ994A3LhxwoRL6cQMoQpZ8tbIMHLSwMWQ6hX0bFoNm1CTSTyf62kDbvdcGESswGd4x6dVLz8tki3_KutxT0xr1YSmUrLman35jh9R4PzRMfRjl0oBV9akHT_Qnu4NZ8_jaf-R8VzkIPVav4aqVDFldPS_tzmGzjc5T0z3YecEGrhswxPVlOSz4p4lwj_ymo8oikwMEFeCFTjMGx0qyPdGUKIqhpV2BD1BjPfL0i2veYqFGJjSdGA2Gr70H5zduAQnpzymdIySLqaezKLA8EYkW6vYZNZoCvtIeZxMfD-l8B9nPspQukoi7WcMrKWcztjgFFrLYolnIKLMstRZqNAmlGHw9HT0tEalkCpGi-fQZnssVrUixmJniou_L3fhkE1eI7MuoVWut3gFB8l7mW_W19Vn_ATxf5tD |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB5qFfSk0opv9-DR2Dx3s0fpgxZrLdpKb2U3mUCgNKVNxZ_vbFKrghdPCYFswmSW-SYz3zcAt3YYGcIlt0LTQuUbyVvF_cCKPRt9ylfQ0XExbEIMBuFkIocVuNtyYRCxaD7De3Na1PLjLFqbX2UNUxOTUuzAbuD7rl2ytb68xw0odP6oGbqhTcmAzUvajutJRzba49fesPlsOrrIRcrVfo1VKaJK5_B_73ME9W96HhtuA88xVHBegxfKKslr2YMRCf9IS0YiyxLWQlwwo8GhZnQomr5XjKAqaxfqEfQE1tsuS7e8pTFmrKVyVYdxpz1qdq3NwAQrJSSTW0pwG2OHJ4GnzFYka4tQJVpJCvxISI5HrhsTAAgTF7nPbcGRdjR6WhOqU9o7geo8m-MpsCDRRuzMF6gjwhhmfjo6UqIQSDmjxjOoGXtMF6UmxnRjivO_L9_Afnf01J_2e4PHCzgw5i_7tC6hmi_XeAV70XuerpbXxSf9BDZrnoo |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2019+27th+European+Signal+Processing+Conference+%28EUSIPCO%29&rft.atitle=Linear+Approximation+of+Deep+Neural+Networks+for+Efficient+Inference+on+Video+Data&rft.au=Rueckauer%2C+Bodo&rft.au=Liu%2C+Shih-Chii&rft.date=2019-09-01&rft.pub=EURASIP&rft.eissn=2076-1465&rft.spage=1&rft.epage=5&rft_id=info:doi/10.23919%2FEUSIPCO.2019.8902997&rft.externalDocID=8902997 |