SHVC CU Processing Aided by a Feedforward Neural Network
The development of multimedia and hardware technologies has led to a great number of industrial video applications, such as virtual reality, high-definition video surveillance, and remote monitoring. As complex communication environments and heterogeneous networks are common in industrial applicatio...
Uložené v:
| Vydané v: | IEEE transactions on industrial informatics Ročník 15; číslo 11; s. 5803 - 5815 |
|---|---|
| Hlavní autori: | , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
Piscataway
IEEE
01.11.2019
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Predmet: | |
| ISSN: | 1551-3203, 1941-0050 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | The development of multimedia and hardware technologies has led to a great number of industrial video applications, such as virtual reality, high-definition video surveillance, and remote monitoring. As complex communication environments and heterogeneous networks are common in industrial applications, industrial videos are required to support a diverse range of display resolutions and transmission channel capacities. Scalable high-efficiency video coding (SHVC) standards provide the tools to meet this requirement. However, it is highly computationally expensive. Coding complexity has a great impact on SHVC performance in industrial applications. Many of these applications are sensitive to time delay and have limited power. Thus, improvements are required to ensure the practical usability of SHVC encoders. In SHVC encoders, intra/interprediction of variable coding unit (CU) sizes is independently performed for the base and enhancement layers (ELs). There are many interlayer similarities that can be exploited to speed up the procedure for EL coding. In this paper, we propose a feedforward neural network aided model for CU size and mode decisions for SHVC, which utilizes base layer coding information and the coding data of spatiotemporal neighboring CUs to decide which CU sizes or prediction modes can be bypassed for certain EL CUs. Two feedforward neural network based learning models are built for CU classification, which are introduced in the procedures for CU size and mode decisions, respectively. According to the analysis from a large number of video sequences, the representative features are directly extracted from the coding information of previously coded neighboring CUs to avoid computational overheads. After the training is finished, these two models are designed and integrated to build classifiers. Then, two online classification approaches are designed for the CU size and mode decision procedures to classify each CU's type. Finally, different candidate CU sizes and prediction modes are adaptively assigned for each type of CU. This approach outperforms the state-of-the-art fast SHVC/high-efficiency video coding (HEVC) algorithms with approximately 19-42% coding time savings or better compression efficiency, which will be beneficial for the realization of real-time scalable video coding. |
|---|---|
| AbstractList | The development of multimedia and hardware technologies has led to a great number of industrial video applications, such as virtual reality, high-definition video surveillance, and remote monitoring. As complex communication environments and heterogeneous networks are common in industrial applications, industrial videos are required to support a diverse range of display resolutions and transmission channel capacities. Scalable high-efficiency video coding (SHVC) standards provide the tools to meet this requirement. However, it is highly computationally expensive. Coding complexity has a great impact on SHVC performance in industrial applications. Many of these applications are sensitive to time delay and have limited power. Thus, improvements are required to ensure the practical usability of SHVC encoders. In SHVC encoders, intra/interprediction of variable coding unit (CU) sizes is independently performed for the base and enhancement layers (ELs). There are many interlayer similarities that can be exploited to speed up the procedure for EL coding. In this paper, we propose a feedforward neural network aided model for CU size and mode decisions for SHVC, which utilizes base layer coding information and the coding data of spatiotemporal neighboring CUs to decide which CU sizes or prediction modes can be bypassed for certain EL CUs. Two feedforward neural network based learning models are built for CU classification, which are introduced in the procedures for CU size and mode decisions, respectively. According to the analysis from a large number of video sequences, the representative features are directly extracted from the coding information of previously coded neighboring CUs to avoid computational overheads. After the training is finished, these two models are designed and integrated to build classifiers. Then, two online classification approaches are designed for the CU size and mode decision procedures to classify each CU's type. Finally, different candidate CU sizes and prediction modes are adaptively assigned for each type of CU. This approach outperforms the state-of-the-art fast SHVC/high-efficiency video coding (HEVC) algorithms with approximately 19-42% coding time savings or better compression efficiency, which will be beneficial for the realization of real-time scalable video coding. |
| Author | An, Ping Feng, Guorui Shen, Liquan |
| Author_xml | – sequence: 1 givenname: Liquan orcidid: 0000-0002-2148-6279 surname: Shen fullname: Shen, Liquan email: jsslq@163.com organization: Shanghai Institute for Advanced Communication and Data Science and the Key Laboratory of Advanced Display and System Application, Shanghai University, Shanghai, China – sequence: 2 givenname: Guorui surname: Feng fullname: Feng, Guorui email: grfeng@shu.edu.cn organization: Key Laboratory of Specialty Fiber Optics and Optical Access Networks, School of Communication and Information Engineering, Shanghai University, Shanghai, China – sequence: 3 givenname: Ping orcidid: 0000-0002-4995-728X surname: An fullname: An, Ping email: anping@shu.edu.cn organization: School of Communication and Information Engineering, Shanghai University, Shanghai, China |
| BookMark | eNp9kMFLwzAUh4NMcJveBS8Fz50vTZsmx1GcGwwV3LyGtHmVztnMpGXsvzdjw4MHD-GXw-977_GNyKC1LRJyS2FCKciH1WIxSYDKSSIpzUFckCGVKY0BMhiEf5bRmCXArsjI-w0Ay4HJIRFv8_ciKtbRq7MVet-0H9G0MWii8hDpaIZoauv22pnoGXuntyG6vXWf1-Sy1luPN-cck_XscVXM4-XL06KYLuMqHNLFtURd61SaJKQQqA2mmteVZlCWKU0lZigNUKzzrBLUCJNKkdCcc11ixZGNyf1p7s7Z7x59pza2d21YqRJGGc8ZZDy0-KlVOeu9w1pVTae7xrad081WUVBHSypYUkdL6mwpgPAH3LnmS7vDf8jdCWkQ8bcuuGRZeD8EOnNL |
| CODEN | ITIICH |
| CitedBy_id | crossref_primary_10_1109_TII_2022_3204681 crossref_primary_10_1109_TBC_2021_3126277 crossref_primary_10_1109_TII_2021_3079231 crossref_primary_10_1007_s11042_024_19179_8 crossref_primary_10_1016_j_nanoen_2024_110186 crossref_primary_10_1109_TII_2022_3151746 crossref_primary_10_1007_s11554_021_01112_z crossref_primary_10_1109_TBC_2022_3222997 |
| Cites_doi | 10.1109/TII.2015.2446769 10.1109/TCSVT.2009.2031520 10.1109/TII.2018.2844214 10.1109/TIP.2017.2740161 10.1109/TCYB.2014.2325594 10.1109/TCSVT.2009.2031387 10.1109/TMM.2015.2510332 10.1109/TCSVT.2014.2302557 10.1109/TII.2018.2801852 10.1109/ICCE.2014.6775968 10.1109/TCSVT.2012.2221191 10.1109/TMM.2016.2524995 10.1186/1687-5281-2013-4 10.1109/TIP.2011.2177849 10.1109/TIP.2016.2601264 10.1109/41.661301 10.1109/TCSVT.2011.2147250 10.1109/TMM.2012.2231060 10.1109/TCSVT.2010.2041825 10.1109/TCSVT.2006.877404 10.1109/WARTIA.2014.6976537 10.1109/ICCE.2014.6775967 10.1109/TII.2015.2491646 10.1109/TBC.2014.2321682 10.1109/TII.2013.2273308 10.1007/BF00994018 10.1109/TIP.2015.2462747 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2019 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2019 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TII.2019.2911708 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1941-0050 |
| EndPage | 5815 |
| ExternalDocumentID | 10_1109_TII_2019_2911708 8693569 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Shanghai Pjiang Program grantid: 15pjd015 – fundername: National Natural Science Foundation of China grantid: 61671282; 61422111 funderid: 10.13039/501100001809 – fundername: Shanghai Shuguang Program grantid: 17SG37 – fundername: Shanghai Science and Technology Innovation Plan grantid: 18010500200 |
| GroupedDBID | 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ IFIPE IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D RIG |
| ID | FETCH-LOGICAL-c291t-f9eafa49d2eaf88eade4a6fca30bb4149e5e9d01ef75c81d8d49821766abec6e3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 9 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000498643600002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1551-3203 |
| IngestDate | Mon Jun 30 10:19:30 EDT 2025 Sat Nov 29 04:16:47 EST 2025 Tue Nov 18 21:39:51 EST 2025 Wed Aug 27 02:44:39 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Issue | 11 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c291t-f9eafa49d2eaf88eade4a6fca30bb4149e5e9d01ef75c81d8d49821766abec6e3 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0002-4995-728X 0000-0002-2148-6279 |
| PQID | 2313673056 |
| PQPubID | 85507 |
| PageCount | 13 |
| ParticipantIDs | ieee_primary_8693569 crossref_citationtrail_10_1109_TII_2019_2911708 proquest_journals_2313673056 crossref_primary_10_1109_TII_2019_2911708 |
| PublicationCentury | 2000 |
| PublicationDate | 2019-11-01 |
| PublicationDateYYYYMMDD | 2019-11-01 |
| PublicationDate_xml | – month: 11 year: 2019 text: 2019-11-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE transactions on industrial informatics |
| PublicationTitleAbbrev | TII |
| PublicationYear | 2019 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref12 ref15 seregin (ref31) 2014 ref14 ref30 ref11 ref10 ref2 ref1 ref17 ref16 ref19 ref18 bjontegaard (ref32) 2001 castillo (ref29) 2006; 7 ref24 ref23 ref26 ref25 ref20 ref22 ref21 ref28 ref27 ref8 ref7 choi (ref5) 2011 ref9 ref4 ref3 gweon (ref6) 2011 |
| References_xml | – ident: ref17 doi: 10.1109/TII.2015.2446769 – ident: ref15 doi: 10.1109/TCSVT.2009.2031520 – volume: 7 start-page: 1159 year: 2006 ident: ref29 article-title: A very fast learning method for neural networks based on sensitivity analysis publication-title: J Mach Learn Res – ident: ref9 doi: 10.1109/TII.2018.2844214 – ident: ref22 doi: 10.1109/TIP.2017.2740161 – ident: ref30 doi: 10.1109/TCYB.2014.2325594 – ident: ref13 doi: 10.1109/TCSVT.2009.2031387 – ident: ref24 doi: 10.1109/TMM.2015.2510332 – ident: ref16 doi: 10.1109/TCSVT.2014.2302557 – year: 2014 ident: ref31 article-title: Common SHM test conditions and software reference configurations – ident: ref1 doi: 10.1109/TII.2018.2801852 – year: 2011 ident: ref6 article-title: Early termination of CU encoding to reduce HEVC complexity – ident: ref21 doi: 10.1109/ICCE.2014.6775968 – ident: ref3 doi: 10.1109/TCSVT.2012.2221191 – ident: ref7 doi: 10.1109/TMM.2016.2524995 – ident: ref26 doi: 10.1186/1687-5281-2013-4 – ident: ref19 doi: 10.1109/TIP.2011.2177849 – ident: ref28 doi: 10.1109/TIP.2016.2601264 – ident: ref2 doi: 10.1109/41.661301 – ident: ref25 doi: 10.1109/TCSVT.2011.2147250 – ident: ref8 doi: 10.1109/TMM.2012.2231060 – ident: ref14 doi: 10.1109/TCSVT.2010.2041825 – year: 2001 ident: ref32 article-title: Calculation of average PSNR difference between RD-curves – ident: ref11 doi: 10.1109/TCSVT.2006.877404 – year: 2011 ident: ref5 article-title: Coding tree pruning based CU early termination, document JCTVC-F092, JCT-VC – ident: ref20 doi: 10.1109/WARTIA.2014.6976537 – ident: ref23 doi: 10.1109/ICCE.2014.6775967 – ident: ref10 doi: 10.1109/TII.2015.2491646 – ident: ref4 doi: 10.1109/TBC.2014.2321682 – ident: ref18 doi: 10.1109/TII.2013.2273308 – ident: ref27 doi: 10.1007/BF00994018 – ident: ref12 doi: 10.1109/TIP.2015.2462747 |
| SSID | ssj0037039 |
| Score | 2.2966397 |
| Snippet | The development of multimedia and hardware technologies has led to a great number of industrial video applications, such as virtual reality, high-definition... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 5803 |
| SubjectTerms | Algorithms Artificial neural networks Classification Coders Coding Complexity Complexity theory Copper Decision analysis Efficiency Encoding Feature extraction Feedforward neural network High definition Industrial applications industrial video Interlayers low-complexity coding mode decision Multimedia Neural networks Remote monitoring Scalability scalable video coding Sequences Streaming media Time compression Time lag Video coding Video compression Video transmission Virtual reality |
| Title | SHVC CU Processing Aided by a Feedforward Neural Network |
| URI | https://ieeexplore.ieee.org/document/8693569 https://www.proquest.com/docview/2313673056 |
| Volume | 15 |
| WOSCitedRecordID | wos000498643600002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1941-0050 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0037039 issn: 1551-3203 databaseCode: RIE dateStart: 20050101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFH9sw4Me_JridEoOXgS7dUnbJMcxHNtlCG6yW8knDGTKPgT_e5O0HYoieGopeaF9L01-L3nv9wBurbFY-NQtroWMEgdhIyEtjVKjnT9CqVSxDMUm6GTC5nP-WIP7XS6MMSYEn5mOvw1n-fpVbf1WWZdlnKQZr0Od0qzI1apmXeJGLg_cqGkvIjgm1ZFkzLvT8djHcPEO5r7OCvu2BIWaKj8m4rC6DI_-917HcFiiSNQvzH4CNbM8hYMv3IJNYE-j5wEazFCZCuAeov5CG43kBxJo6FYth1d9zCzyBB2ut0kREX4Gs-HDdDCKyjIJkXIftIksN8KKhGvsroz5COhEZFYJEkuZOA_IpIbruGcsTZWDp0wnnGFPDCmcATNDzqGxfF2aC0BcYeLwnLTKSRppuSUxFj0msRUypb0WdCvN5arkEPelLF7y4EvEPHe6zr2u81LXLbjbSbwV_Bl_tG163e7alWptQbsyTl7-YOvcwVKSUe__XP4udQX7vu8ibbANjc1qa65hT71vFuvVTRg7n9V5wXQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFD7MKagP3sXp1Dz4ItgtTXpJHsdwbDiH4CZ7K0mawEA22UXw35v0MhRF8Kml5LTpd9LkO825ANwYbYhwoVs8FdILLIX1hDSxF-rU2iNxLBWWWbGJeDBg4zF_qsDdOhZGa505n-mGO8328tOZWrlfZU0WcRpGfAM2wyAgOI_WKuddascuz7Kjhr5HCablpiTmzWGv57y4eINwV2mFfVuEsqoqP6bibH3p7P-vZwewV_BI1MoVfwgVPT2C3S_ZBY-BPXdf2qg9QkUwgL2IWpNUp0h-IIE6dt2yjNV5zSKXosPebZD7hJ_AqHM_bHe9olCCp-wLLT3DtTAi4CmxR8acD3QgIqMExVIG1gbSoeYp9rWJQ2UJKksDzohLDSmsCiNNT6E6nU31GSCuCLWMThplJbU03FBMhM8kMUKGsV-DZolcooos4q6YxWuSWROYJxbrxGGdFFjX4HYt8ZZn0Pij7bHDdt2ugLUG9VI5SfGJLRJLTGkUOwvo_Hepa9juDh_7Sb83eLiAHfecPIiwDtXlfKUvYUu9LyeL-VU2jj4BEtjEuw |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=SHVC+CU+Processing+Aided+by+a+Feedforward+Neural+Network&rft.jtitle=IEEE+transactions+on+industrial+informatics&rft.au=Shen%2C+Liquan&rft.au=Feng%2C+Guorui&rft.au=An%2C+Ping&rft.date=2019-11-01&rft.issn=1551-3203&rft.eissn=1941-0050&rft.volume=15&rft.issue=11&rft.spage=5803&rft.epage=5815&rft_id=info:doi/10.1109%2FTII.2019.2911708&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TII_2019_2911708 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1551-3203&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1551-3203&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1551-3203&client=summon |