Rate-Distortion Optimal Joint Texture and Depth Map Coding for 3-D Video Streaming

For high compression efficiency, 3-D video coding usually employs a multimode methodology to exploit the dependencies between multiple views as well as between texture and depth. However, different coding modes will posses differentiating error propagation behaviour when the compressed 3-D video bit...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE transactions on multimedia Jg. 22; H. 3; S. 610 - 625
Hauptverfasser:	Gao, Pan, Paul, Manoranjan
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	Piscataway IEEE 01.03.2020 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	3-D video transmission Algorithms Coding Distortion Dynamic programming Encoding Error resilience joint texture and depth map coding Lagrange multiplier Optimization Packet loss Propagation modes Rate-distortion rate-distortion optimization Statistical models Streaming media Switching theory Synthesis Texture variable-block-size prediction Video coding Video compression Video transmission
ISSN:	1520-9210, 1941-0077
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	For high compression efficiency, 3-D video coding usually employs a multimode methodology to exploit the dependencies between multiple views as well as between texture and depth. However, different coding modes will posses differentiating error propagation behaviour when the compressed 3-D video bit stream is transmitted over packet-switched networks, and thus lead to different amount of visual distortions. Further, the texture and depth distortions are combined in a highly complex fashion to produce the overall view synthesis distortion. To minimize the expected view synthesis distortion, this paper proposes an efficient rate-distortion optimized algorithm for joint selection of texture and depth modes. Firstly, a statistical model is developed to estimate the overall view synthesis distortion, in which the channel distortions caused by error propagation under different coding modes are analyzed. Then, joint optimization of texture and depth modes is derived within an operational rate-distortion framework using the Lagrange multiplier method. The adjacent block dependency caused by warping operation is explicitly considered in optimization, for which we develop a dynamic programming method to find the optimal solution. Finally, we extend the Lagrange minimization method to the more general variable-block-size prediction case, where the optimal quadtree tree structure and the combined coding modes are jointly determined using a multi-level dual trellis. Experimental results are presented for a wide range of packet loss rates to illustrate the effectiveness of the proposed algorithm.
AbstractList	For high compression efficiency, 3-D video coding usually employs a multimode methodology to exploit the dependencies between multiple views as well as between texture and depth. However, different coding modes will posses differentiating error propagation behaviour when the compressed 3-D video bit stream is transmitted over packet-switched networks, and thus lead to different amount of visual distortions. Further, the texture and depth distortions are combined in a highly complex fashion to produce the overall view synthesis distortion. To minimize the expected view synthesis distortion, this paper proposes an efficient rate-distortion optimized algorithm for joint selection of texture and depth modes. Firstly, a statistical model is developed to estimate the overall view synthesis distortion, in which the channel distortions caused by error propagation under different coding modes are analyzed. Then, joint optimization of texture and depth modes is derived within an operational rate-distortion framework using the Lagrange multiplier method. The adjacent block dependency caused by warping operation is explicitly considered in optimization, for which we develop a dynamic programming method to find the optimal solution. Finally, we extend the Lagrange minimization method to the more general variable-block-size prediction case, where the optimal quadtree tree structure and the combined coding modes are jointly determined using a multi-level dual trellis. Experimental results are presented for a wide range of packet loss rates to illustrate the effectiveness of the proposed algorithm.
Author	Gao, Pan Paul, Manoranjan
Author_xml	– sequence: 1 givenname: Pan orcidid: 0000-0002-4492-5430 surname: Gao fullname: Gao, Pan email: gaopan.1005@gmail.com organization: College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China – sequence: 2 givenname: Manoranjan orcidid: 0000-0001-6870-5056 surname: Paul fullname: Paul, Manoranjan email: mpaul@csu.edu.au organization: School of Computing and Mathematics, Charles Sturt University, Bathurst, NSW, Australia
BookMark	eNp9kE1LAzEQhoNUsK3eBS8Bz1szye5mc5TWT1oKtXoN2W6iKe1mTVLQf29KiwcPzmWGmXnnZZ4B6rWu1QhdAhkBEHGznM1GlIAYUcFSlCeoDyKHjBDOe6kuKMkEBXKGBiGsCYG8ILyPFgsVdTaxITofrWvxvIt2qzb42dk24qX-ijuvsWobPNFd_MAz1eGxa2z7jo3zmGUT_GYb7fBL9FptU_8cnRq1CfrimIfo9f5uOX7MpvOHp_HtNFtRATHjjAKYktQNBVZTWjNaMSqMzlXOVNVUxjRGMeCCp1mtSt0Uoq7KOi9oXXJgQ3R9uNt597nTIcq12_k2WUrKSppDkVOWtshha-VdCF4b2fn0oP-WQOSenEzk5J6cPJJLkvKPZGWj2tOJXtnNf8Krg9BqrX99Ki5IBQX7AQSDey0
CODEN	ITMUF8
CitedBy_id	crossref_primary_10_1109_TIP_2020_3036760
Cites_doi	10.1109/TMM.2005.864313 10.1109/TCSVT.2007.903665 10.1109/TCSVT.2008.2005802 10.1109/TCSVT.2015.2477935 10.1109/29.90373 10.1109/TCSVT.2014.2352571 10.1016/j.jvcir.2012.04.006 10.1109/PCS.2012.6213284 10.1109/ICIP.2017.8296613 10.1109/JSTSP.2015.2402633 10.1109/TMM.2014.2331013 10.1109/TCSVT.2014.2363746 10.1109/TIP.2013.2269274 10.1109/TCSVT.2011.2129270 10.1109/TIP.2004.826089 10.1109/TMM.2014.2299768 10.1109/TIP.2017.2690058 10.1109/JPROC.2010.2091090 10.1109/TCSVT.2018.2844743 10.1109/ICASSP.2015.7178210 10.1109/TIP.2011.2118218 10.1109/TMM.2017.2694218 10.1109/83.265978 10.1109/TCSVT.2018.2877903 10.1109/TCSVT.2016.2631568 10.1109/TMM.2017.2766043 10.1109/TIP.2009.2032895 10.1109/TCSVT.2015.2407791 10.1109/TIP.2016.2535345 10.1109/GLOCOM.2016.7841779 10.1109/49.848250 10.1109/TIP.2011.2168411 10.1109/TCSVT.2016.2527318 10.1109/TCSVT.2014.2380191 10.1109/TCSVT.2003.814966
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2020
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/TMM.2019.2933336
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering Computer Science
EISSN	1941-0077
EndPage	625
ExternalDocumentID	10_1109_TMM_2019_2933336 8790815
Genre	orig-research
GrantInformation_xml	– fundername: Natural Science Foundation of Jiangsu Province grantid: BK20170806 funderid: 10.13039/501100004608 – fundername: National Natural Science Foundation of China grantid: 61701227 funderid: 10.13039/501100001809 – fundername: Australian Research Council grantid: DP190102574 funderid: 10.13039/501100000923
GroupedDBID	-~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 IFIPE IFJZH IPLJI JAVBF LAI M43 O9- OCL P2P PQQKQ RIA RIE RNS TN5 VH1 ZY4 AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c291t-73211f60bd213b22b328329fe4a43a8d8ffdfa317972b3ba6ed59b86b452b6713
IEDL.DBID	RIE
ISICitedReferencesCount	3
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000519576700004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1520-9210
IngestDate	Sun Nov 30 05:27:25 EST 2025 Tue Nov 18 22:17:19 EST 2025 Sat Nov 29 03:10:03 EST 2025 Wed Aug 27 02:42:22 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	3
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c291t-73211f60bd213b22b328329fe4a43a8d8ffdfa317972b3ba6ed59b86b452b6713
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-4492-5430 0000-0001-6870-5056
PQID	2362415423
PQPubID	75737
PageCount	16
ParticipantIDs	ieee_primary_8790815 proquest_journals_2362415423 crossref_primary_10_1109_TMM_2019_2933336 crossref_citationtrail_10_1109_TMM_2019_2933336
PublicationCentury	2000
PublicationDate	2020-03-01
PublicationDateYYYYMMDD	2020-03-01
PublicationDate_xml	– month: 03 year: 2020 text: 2020-03-01 day: 01
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE transactions on multimedia
PublicationTitleAbbrev	TMM
PublicationYear	2020
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref34 (ref39) 2010 ref37 ref15 ref36 ref14 ref31 chang (ref32) 0 ref33 ref11 ref10 ref2 ref1 ref17 ref16 (ref40) 2014 lin (ref13) 2011; 21 ref18 (ref46) 2016 li (ref44) 2003 macchiavello (ref19) 0 ref24 ref23 abreu (ref35) 0 bjontegaard (ref45) 2001 ref47 ref25 ref20 ref41 ref22 (ref38) 2012 ref21 ref43 ref28 ref27 fehn (ref26) 0 zhu (ref12) 2009; 19 ref29 ref8 tian (ref30) 2013 ref7 ref9 ref4 ref3 wenger (ref42) 1999 ref6 ref5
References_xml	– ident: ref10 doi: 10.1109/TMM.2005.864313 – ident: ref3 doi: 10.1109/TCSVT.2007.903665 – start-page: 678 year: 0 ident: ref32 article-title: Joint souce-channel rate-distortion optimization with motion information sharing for H.264/AVC video-plus-depth coding publication-title: Proc Asilomar Conf Signals Syst Comput – volume: 19 start-page: 3 year: 2009 ident: ref12 article-title: Error resilient video coding using redundant pictures publication-title: IEEE Trans Circuits Syst Video Technol doi: 10.1109/TCSVT.2008.2005802 – ident: ref6 doi: 10.1109/TCSVT.2015.2477935 – ident: ref34 doi: 10.1109/29.90373 – ident: ref4 doi: 10.1109/TCSVT.2014.2352571 – ident: ref14 doi: 10.1016/j.jvcir.2012.04.006 – ident: ref8 doi: 10.1109/PCS.2012.6213284 – ident: ref22 doi: 10.1109/ICIP.2017.8296613 – ident: ref23 doi: 10.1109/JSTSP.2015.2402633 – ident: ref21 doi: 10.1109/TMM.2014.2331013 – year: 1999 ident: ref42 article-title: Proposed error patterns for internet experiments – year: 2001 ident: ref45 article-title: Calculation of average PSNR differences between RD curves – ident: ref16 doi: 10.1109/TCSVT.2014.2363746 – year: 2010 ident: ref39 article-title: 3DV/FTV EE2: Report on VSRS Extrapolation – ident: ref5 doi: 10.1109/TIP.2013.2269274 – volume: 21 start-page: 589 year: 2011 ident: ref13 article-title: Multiple description coding for H.264/AVC with redundancy allocation at macro block level publication-title: IEEE Trans Circuits Syst Video Technol doi: 10.1109/TCSVT.2011.2129270 – ident: ref29 doi: 10.1109/TIP.2004.826089 – ident: ref20 doi: 10.1109/TMM.2014.2299768 – start-page: 113 year: 0 ident: ref35 article-title: Optimal Lagrange multipliers for dependent rate allocation in video coding publication-title: Signal Process Image Commun – ident: ref18 doi: 10.1109/TIP.2017.2690058 – ident: ref1 doi: 10.1109/JPROC.2010.2091090 – ident: ref17 doi: 10.1109/TCSVT.2018.2844743 – year: 2003 ident: ref44 article-title: Adaptive rate control with HRD consideration – ident: ref31 doi: 10.1109/ICASSP.2015.7178210 – ident: ref27 doi: 10.1109/TIP.2011.2118218 – ident: ref9 doi: 10.1109/TMM.2017.2694218 – ident: ref37 doi: 10.1109/83.265978 – ident: ref24 doi: 10.1109/TCSVT.2018.2877903 – year: 2014 ident: ref40 article-title: Common test conditions of 3DV core experiments – ident: ref36 doi: 10.1109/TCSVT.2016.2631568 – ident: ref25 doi: 10.1109/TMM.2017.2766043 – ident: ref11 doi: 10.1109/TIP.2009.2032895 – year: 2016 ident: ref46 – ident: ref47 doi: 10.1109/TCSVT.2015.2407791 – start-page: 83050c-1 year: 0 ident: ref19 article-title: Reference frame selection for loss-resilient depth map coding in multiview video conferencing publication-title: Proc SPIE Vis Inf Process Commun – ident: ref28 doi: 10.1109/TIP.2016.2535345 – ident: ref15 doi: 10.1109/GLOCOM.2016.7841779 – ident: ref43 doi: 10.1109/49.848250 – ident: ref33 doi: 10.1109/TIP.2011.2168411 – year: 2012 ident: ref38 – year: 2013 ident: ref30 article-title: CE1.h: Backward view synthesis prediction using neighboring blocks – ident: ref2 doi: 10.1109/TCSVT.2016.2527318 – ident: ref7 doi: 10.1109/TCSVT.2014.2380191 – ident: ref41 doi: 10.1109/TCSVT.2003.814966 – start-page: 93 year: 0 ident: ref26 article-title: Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3-D-TV publication-title: Proc SPIE Stereoscopic Displays Virtual Reality Syst VI
SSID	ssj0014507
Score	2.2945235
Snippet	For high compression efficiency, 3-D video coding usually employs a multimode methodology to exploit the dependencies between multiple views as well as between...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	610
SubjectTerms	3-D video transmission Algorithms Coding Distortion Dynamic programming Encoding Error resilience joint texture and depth map coding Lagrange multiplier Optimization Packet loss Propagation modes Rate-distortion rate-distortion optimization Statistical models Streaming media Switching theory Synthesis Texture variable-block-size prediction Video coding Video compression Video transmission
Title	Rate-Distortion Optimal Joint Texture and Depth Map Coding for 3-D Video Streaming
URI	https://ieeexplore.ieee.org/document/8790815 https://www.proquest.com/docview/2362415423
Volume	22
WOSCitedRecordID	wos000519576700004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1941-0077 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014507 issn: 1520-9210 databaseCode: RIE dateStart: 19990101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB60eNCDVatYX-zBi-DaPDaPPYpVRKyK1OIt7CYTLNiktKm_39ltWhRFMKeQ7IaQL7vfNzs7MwCnJlUhiTbkyleSizxzObGgx1E7At00QLTh0YP76OEhfn2VTytwvoyFQUS7-QwvzKn15WdlOjNLZZ04ksRgwSqsRlE4j9VaegxEYEOjiY4cLsmOWbgkHdnp93pmD5e8IGqjI_xGQbamyo-J2LLLTfN_77UFm7WKZJdz2LdhBYsdaC4qNLB6wO7Axpd0gy14fiZhybs2MYjBgz3SfDGi59yVw6JifZqnZxNkqshYF8fVG-upMbsqDbsx0rbM5102GGZYMuPLViO6vgsvN9f9q1te11TgqSfdikc-WXx56OjMc33tedo3tYpkjkIJX8VZnOdZrkhUyIjuaRViFkgdh1oEng7Jot2DRlEWuA9MoFZurFER6wuRR7H2nVT4aapdJxca29BZfOYkrROOm7oX74k1PByZEDCJASapgWnD2bLHeJ5s44-2LQPEsl2NQRuOFkgm9WicJh6xNAkVUo4Hv_c6hHXP2NF2b9kRNKrJDI9hLf2ohtPJif3RPgHt6M6y
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dT9swED8xNmnbA-VrWseXH3hBwjRxnA8_IjrUQVsmVBBvkZ1cRKU1qUq6v39nN61AoEnkKUrsKMov9u93Pt8dwLFNVUiiDbkOtOKyyH1OLCg4Gk-in4WILjz6vh8Ph8nDg_q9BqerWBhEdJvP8MyeOl9-XmVzu1TWSWJFDBZ-gI-hlMJbRGutfAYydMHRREgeV2TJLJ2SnuqMBgO7i0udEbnREb0gIVdV5dVU7PjlsvW-N9uEjUZHsvMF8FuwhuU2tJY1GlgzZLfh67OEgztwe0vSknddahCLCLuhGWNCz7mqxmXNRjRTz2fIdJmzLk7rRzbQU3ZRWX5jpG5ZwLvsfpxjxaw3W0_o-i7cXf4cXfR4U1WBZ0L5NY8DsvmKyDO58AMjhAlstSJVoNQy0EmeFEVeaJIVKqZ7RkeYh8okkZGhMBHZtN9gvaxK_A5MotF-YlAT70tZxIkJvEwGWWZ8r5AG29BZfuY0a1KO28oXf1JnengqJWBSC0zaANOGk1WP6SLdxn_a7lggVu0aDNqwv0QybcbjUyqIp0mqkHb88XavI_jcGw36af_X8HoPvghrVbudZvuwXs_meACfsr_1-Gl26H66f9lE0fk
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Rate-Distortion+Optimal+Joint+Texture+and+Depth+Map+Coding+for+3-D+Video+Streaming&rft.jtitle=IEEE+transactions+on+multimedia&rft.au=Gao%2C+Pan&rft.au=Paul%2C+Manoranjan&rft.date=2020-03-01&rft.issn=1520-9210&rft.eissn=1941-0077&rft.volume=22&rft.issue=3&rft.spage=610&rft.epage=625&rft_id=info:doi/10.1109%2FTMM.2019.2933336&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TMM_2019_2933336
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1520-9210&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1520-9210&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1520-9210&client=summon