Landmarking for Navigational Streaming of Stored High-Dimensional Media

Modern media data such as 360° videos and light field (LF) images are typically captured in much higher dimensions than the observers' visual displays. To efficiently browse high-dimensional media, a navigational streaming model is considered: a client navigates the media space by dictating a n...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE transactions on circuits and systems for video technology Jg. 32; H. 8; S. 5663 - 5679
Hauptverfasser:	Yuan, Yuan, Cheung, Gene, Frossard, Pascal, Zhao, H. Vicky, Huang, Jiwu
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	New York IEEE 01.08.2022 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Branch and bound methods Costs distributed source coding Encoding Media media compression Navigation Navigational streaming Optimization Random access Servers Storage Streaming media Videos Visual observation
ISSN:	1051-8215, 1558-2205
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	Modern media data such as 360° videos and light field (LF) images are typically captured in much higher dimensions than the observers' visual displays. To efficiently browse high-dimensional media, a navigational streaming model is considered: a client navigates the media space by dictating a navigation path to a server, who in response transmits the corresponding pre-encoded media data units (MDU) to the client one-by-one in sequence. Assuming that the MDU quality is pre-chosen and fixed, the problem resides in selecting and storing redundant representations of MDUs at the server in order to best trade off storage and transmission costs, while enabling adequate user's random access. We address this problem with a landmark-based MDU optimization framework. The media space is divided into neighborhoods, each containing one landmark (a chosen MDU). MDUs in a neighborhood use the associated landmark as a predictor for inter-coding. Thus, for any MDU transition within the same neighborhood, only one inter-coded MDU transmission is required when the landmark resides in the decoder buffer. It results in lower transmission cost and enables navigational random access. To optimize an MDU structure, we employ tree-structured vector quantizer (TSVQ) to first optimize landmark locations, then iteratively add P-MDUs as refinements using a fast branch-and-bound technique. Taking interactive LF images and viewport adaptive 360° images as illustrative applications, and I-, P- and previously proposed merge frames to intra- and inter-code MDUs, we show experimentally that landmarked MDU structures can noticeably reduce the expected transmission cost compared with MDU structures without landmarks.
AbstractList	Modern media data such as 360° videos and light field (LF) images are typically captured in much higher dimensions than the observers’ visual displays. To efficiently browse high-dimensional media, a navigational streaming model is considered: a client navigates the media space by dictating a navigation path to a server, who in response transmits the corresponding pre-encoded media data units (MDU) to the client one-by-one in sequence. Assuming that the MDU quality is pre-chosen and fixed, the problem resides in selecting and storing redundant representations of MDUs at the server in order to best trade off storage and transmission costs, while enabling adequate user’s random access. We address this problem with a landmark-based MDU optimization framework. The media space is divided into neighborhoods, each containing one landmark (a chosen MDU). MDUs in a neighborhood use the associated landmark as a predictor for inter-coding. Thus, for any MDU transition within the same neighborhood, only one inter-coded MDU transmission is required when the landmark resides in the decoder buffer. It results in lower transmission cost and enables navigational random access. To optimize an MDU structure, we employ tree-structured vector quantizer (TSVQ) to first optimize landmark locations, then iteratively add P-MDUs as refinements using a fast branch-and-bound technique. Taking interactive LF images and viewport adaptive 360° images as illustrative applications, and I-, P- and previously proposed merge frames to intra- and inter-code MDUs, we show experimentally that landmarked MDU structures can noticeably reduce the expected transmission cost compared with MDU structures without landmarks.
Author	Zhao, H. Vicky Huang, Jiwu Cheung, Gene Yuan, Yuan Frossard, Pascal
Author_xml	– sequence: 1 givenname: Yuan orcidid: 0000-0003-3352-0662 surname: Yuan fullname: Yuan, Yuan email: yuanyustc@hotmail.com organization: School of Computer Science, Guangdong Polytechnic Normal University, Guangzhou, China – sequence: 2 givenname: Gene orcidid: 0000-0002-5571-4137 surname: Cheung fullname: Cheung, Gene email: genec@yorku.ca organization: Department of EECS, York University, ON, Toronto, Canada – sequence: 3 givenname: Pascal orcidid: 0000-0002-4010-714X surname: Frossard fullname: Frossard, Pascal email: pascal.frossard@epfl.ch organization: Signal Processing Laboratory (LTS4), École Polytechnique Fédérale de Lausanne (EPFL), CH, Switzerland – sequence: 4 givenname: H. Vicky orcidid: 0000-0002-3690-9924 surname: Zhao fullname: Zhao, H. Vicky email: vzhao@tsinghua.edu.cn organization: Department of Automation, Tsinghua University, Beijing, China – sequence: 5 givenname: Jiwu orcidid: 0000-0002-7625-5689 surname: Huang fullname: Huang, Jiwu email: jwhuang@szu.edu.cn organization: Guangdong Key Laboratory of Intelligent Information Processing and Shenzhen Key Laboratory of Media Security, Shenzhen University, Shenzhen, China
BookMark	eNo9kE1PAjEQhhuDiaD-Ab1s4nmxH9ttezSoYIJ6AL023XYWi7DFdjHx37u4xFOnmeedzDwjNGhCAwhdETwmBKvb5WTxvhxTTOmYEY6FxCdoSDiXOaWYD7oac5JLSvgZGqW0xpgUshBDNJ2bxm1N_PTNKqtDzF7Mt1-Z1ofGbLJFG8FsD61Qd58QwWUzv_rI7_0WmtRDz-C8uUCntdkkuDy-5-jt8WE5meXz1-nT5G6eW6pwm1dQCecIrStmGSulqCQRQKuywApk7Sy2rmRcGlUSUzsQvMJFaa2olIUOZOfopp-7i-FrD6nV67CP3RpJ01IJLhhlvKNoT9kYUopQ61303ZU_mmB9EKb_hOmDMH0U1oWu-5AHgP-AElgJydkvmnZpXg
CODEN	ITCTEM
Cites_doi	10.1117/12.527327 10.1109/ICC.2017.7996611 10.1109/ICME.2011.6011865 10.1109/ICIP.2017.8296617 10.1117/1.601531 10.1109/PCS.2009.5167460 10.1109/TC.1977.1674939 10.1016/j.image.2021.116202 10.1109/MMUL.2018.011921238 10.1109/PACKET.2009.5152147 10.1145/3123266.3123291 10.1109/TCSVT.2021.3055985 10.1007/978-1-4615-3626-0 10.1109/ICASSP.2012.6288163 10.1109/TCSVT.2020.3046242 10.1109/ICIP.2019.8803668 10.1109/TCSVT.2003.814969 10.1109/TCSVT.2012.2221191 10.1109/MMSP.2004.1436558 10.1109/TIP.2010.2070074 10.1109/TVCG.2018.2793599 10.1109/TIP.2016.2571564 10.1109/ICIP.2009.5414623 10.1109/TCSVT.2018.2886805 10.1109/TCSVT.2003.817626 10.5594/M001787 10.1109/TMM.2019.2932614 10.1109/ICIP.2016.7532582 10.1109/TMM.2007.893350 10.3390/sym12091491 10.1109/TMM.2020.2987682 10.1049/el:19961075 10.1109/ICIP.2017.8296676
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/TCSVT.2022.3150780
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE/IET Electronic Library CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1558-2205
EndPage	5679
ExternalDocumentID	10_1109_TCSVT_2022_3150780 9709785
Genre	orig-research
GrantInformation_xml	– fundername: National Science Foundations of China grantid: U19B2022; U1636202; 61701310 funderid: 10.13039/501100001809 – fundername: NSERC grantid: RGPIN-2019-06271; RGPAS-2019-00110 funderid: 10.13039/501100000038 – fundername: Guangdong Natural Science Foundation grantid: 2020A1515110781 funderid: 10.13039/501100003453
GroupedDBID	-~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 ICLAB IFIPE IFJZH IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS RXW TAE TN5 VH1 AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c290t-beb7dd12fb3c33687b817e2b6409e8fdc0cd6358a961afde75b046cc7b9ce7e23
IEDL.DBID	RIE
ISICitedReferencesCount	0
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000835828500059&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1051-8215
IngestDate	Sun Nov 30 04:33:41 EST 2025 Sat Nov 29 01:44:18 EST 2025 Wed Aug 27 02:23:49 EDT 2025
IsDoiOpenAccess	false
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	8
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c290t-beb7dd12fb3c33687b817e2b6409e8fdc0cd6358a961afde75b046cc7b9ce7e23
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-3690-9924 0000-0002-5571-4137 0000-0002-4010-714X 0000-0003-3352-0662 0000-0002-7625-5689
OpenAccessLink	http://infoscience.epfl.ch/record/295939
PQID	2697573235
PQPubID	85433
PageCount	17
ParticipantIDs	proquest_journals_2697573235 crossref_primary_10_1109_TCSVT_2022_3150780 ieee_primary_9709785
PublicationCentury	2000
PublicationDate	2022-08-01
PublicationDateYYYYMMDD	2022-08-01
PublicationDate_xml	– month: 08 year: 2022 text: 2022-08-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on circuits and systems for video technology
PublicationTitleAbbrev	TCSVT
PublicationYear	2022
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref35 ref12 ref34 ref37 ref14 Van der Auwera (ref16) 2020 ref36 ref31 ref11 ref33 ref10 ref2 ref17 ref39 ref19 ref18 Bross (ref38) 2018 ref24 ref23 ref26 ref25 Rerabek (ref1) ref20 ref22 ref21 ref28 ref27 ref29 ref8 ref7 Van der Auwera (ref15) 2017 ref9 ref4 ref3 ref6 ref5 Ng (ref30) 2005; 2 Bartelmess (ref32) 2016
References_xml	– ident: ref21 doi: 10.1117/12.527327 – ident: ref34 doi: 10.1109/ICC.2017.7996611 – ident: ref7 doi: 10.1109/ICME.2011.6011865 – ident: ref31 doi: 10.1109/ICIP.2017.8296617 – ident: ref17 doi: 10.1117/1.601531 – ident: ref26 doi: 10.1109/PCS.2009.5167460 – ident: ref18 doi: 10.1109/TC.1977.1674939 – volume-title: AHG8: TSP Evaluation With Viewport-Aware Quality Metric for 360 Video year: 2017 ident: ref15 – ident: ref39 doi: 10.1016/j.image.2021.116202 – ident: ref2 doi: 10.1109/MMUL.2018.011921238 – ident: ref3 doi: 10.1109/PACKET.2009.5152147 – volume-title: Versatile Video Coding (Draft 1) year: 2018 ident: ref38 – ident: ref33 doi: 10.1145/3123266.3123291 – ident: ref13 doi: 10.1109/TCSVT.2021.3055985 – ident: ref20 doi: 10.1007/978-1-4615-3626-0 – ident: ref8 doi: 10.1109/ICASSP.2012.6288163 – volume: 2 start-page: 1 issue: 11 volume-title: Comput. Sci. Tech. Rep. year: 2005 ident: ref30 article-title: Light field photography with a hand-held plenoptic camera – ident: ref12 doi: 10.1109/TCSVT.2020.3046242 – ident: ref24 doi: 10.1109/ICIP.2019.8803668 – ident: ref28 doi: 10.1109/TCSVT.2003.814969 – ident: ref37 doi: 10.1109/TCSVT.2012.2221191 – ident: ref6 doi: 10.1109/MMSP.2004.1436558 – ident: ref5 doi: 10.1109/TIP.2010.2070074 – ident: ref35 doi: 10.1109/TVCG.2018.2793599 – ident: ref19 doi: 10.1109/TIP.2016.2571564 – ident: ref4 doi: 10.1109/ICIP.2009.5414623 – ident: ref10 doi: 10.1109/TCSVT.2018.2886805 – ident: ref23 doi: 10.1109/TCSVT.2003.817626 – ident: ref14 doi: 10.5594/M001787 – ident: ref25 doi: 10.1109/TMM.2019.2932614 – start-page: 1 volume-title: Proc. 8th Int. Conf. Quality Multimedia Exper. (QoMEX) ident: ref1 article-title: New light field image dataset – ident: ref9 doi: 10.1109/ICIP.2016.7532582 – volume-title: Compression efficiency of different picture coding structures in high efficiency video coding (HEVC) year: 2016 ident: ref32 – volume-title: Viewport-aware quality metric for 360-degree video year: 2020 ident: ref16 – ident: ref22 doi: 10.1109/TMM.2007.893350 – ident: ref11 doi: 10.3390/sym12091491 – ident: ref36 doi: 10.1109/TMM.2020.2987682 – ident: ref29 doi: 10.1049/el:19961075 – ident: ref27 doi: 10.1109/ICIP.2017.8296676
SSID	ssj0014847
Score	2.3848026
Snippet	Modern media data such as 360° videos and light field (LF) images are typically captured in much higher dimensions than the observers' visual displays. To... Modern media data such as 360° videos and light field (LF) images are typically captured in much higher dimensions than the observers’ visual displays. To...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Index Database Publisher
StartPage	5663
SubjectTerms	Branch and bound methods Costs distributed source coding Encoding Media media compression Navigation Navigational streaming Optimization Random access Servers Storage Streaming media Videos Visual observation
Title	Landmarking for Navigational Streaming of Stored High-Dimensional Media
URI	https://ieeexplore.ieee.org/document/9709785 https://www.proquest.com/docview/2697573235
Volume	32
WOSCitedRecordID	wos000835828500059&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-2205 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014847 issn: 1051-8215 databaseCode: RIE dateStart: 19910101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV05T8MwGP3UVgwwcBVEuZSBDUwTu75GVCgMqEKioG6Rr0gMTVGv34_tpFUlWNgSxUmsZyd-z98FcCO8qODMaUSVYahnLEOKkgI5zJSkhRA0NbHYBB8OxXgs3xpwt4mFcc5F5zN3Hw6jLd9OzTJslXUlD1EHtAlNznkVq7WxGPRELCbm6UKGfAfoOkAmld1R__1z5KUgxl6hev4TUkBuLUKxqsqvX3FcXwYH_-vZIezXPDJ5qAb-CBquPIa9reyCbXh-VaWdqLgZnnhumgzVKibUCOQ7CeZoNQmXpoU_8S-0SXD6QI8h33-VqyMJZhx1Ah-Dp1H_BdV1E5DBMl0g7TS3NsOFJoYQJrgWGXdYM6_lnCisSf2QECqUZJkqrONUe5VsDNfSON-QnEKrnJbuDBLKnQeWKIEL0bPGk0lPaVSoDCG5SxXpwO0ayPy7So-RR1mRyjzCngfY8xr2DrQDdJuWNWoduFxjn9df0DzHTHLKCSb0_O-7LmA3PLtyxruE1mK2dFewY1aLr_nsOk6OH3Ekt24
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PT8MgFH6Z00Q9-Gsap1N78Ka4FkqBo5nOGedi4jS7NRRo4mGb2a-_X6DdskQv3tqUBvJBy_d4730P4Jpbo4IlJkNUqgTFSidIUpIjgxMpaM45DZUvNsF6PT4YiLcK3K5yYYwxPvjM3LlL78vXYzV3R2VNwVzWAd2ATRrHOCqytVY-g5j7cmKWMETIDoEuU2RC0ey33j_71hjE2NqolgE5Eci1bcjXVfn1M_Y7THv_f2M7gL2SSQb3xdQfQsWMjmB3TV-wBk9dOdJD6Y_DA8tOg55ceEkNR78D55CWQ_donNsb26EOXNgHenCK_4VaR-AcOfIYPtqP_VYHlZUTkMIinKHMZEzrCOcZUYQknGU8YgZnibXmDM-1Cu2kEMqlSCKZa8NoZu1kpVgmlLENyQlUR-OROYWAMmOBJZLjnMdaWTppSY10tSEEM6EkdbhZApl-FwIZqTcsQpF62FMHe1rCXoeag27VskStDo0l9mn5DU1TnAhGGcGEnv391hVsd_qv3bT73Hs5hx3XTxGa14DqbDI3F7ClFrOv6eTSL5QfKb26tQ
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Landmarking+for+Navigational+Streaming+of+Stored+High-Dimensional+Media&rft.jtitle=IEEE+transactions+on+circuits+and+systems+for+video+technology&rft.au=Yuan%2C+Yuan&rft.au=Cheung%2C+Gene&rft.au=Frossard%2C+Pascal&rft.au=Zhao%2C+H.+Vicky&rft.date=2022-08-01&rft.issn=1051-8215&rft.eissn=1558-2205&rft.volume=32&rft.issue=8&rft.spage=5663&rft.epage=5679&rft_id=info:doi/10.1109%2FTCSVT.2022.3150780&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TCSVT_2022_3150780
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1051-8215&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1051-8215&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1051-8215&client=summon