Power, Performance, and Image Quality Tradeoffs in Foveated Rendering
Extended reality (XR) devices, including augmented, virtual, and mixed reality, provide a deeply immersive experience. However, practical limitations like weight, heat, and comfort put extreme constraints on the performance, power consumption, and image quality of such systems. In this paper, we stu...
Uložené v:
| Vydané v: | Proceedings (IEEE Conference on Virtual Reality and 3D User Interfaces. Online) s. 205 - 214 |
|---|---|
| Hlavní autori: | , , , , , , |
| Médium: | Konferenčný príspevok.. |
| Jazyk: | English |
| Vydavateľské údaje: |
IEEE
01.03.2023
|
| Predmet: | |
| ISSN: | 2642-5254 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Extended reality (XR) devices, including augmented, virtual, and mixed reality, provide a deeply immersive experience. However, practical limitations like weight, heat, and comfort put extreme constraints on the performance, power consumption, and image quality of such systems. In this paper, we study how these constraints form the tradeoff between Fixed Foveated Rendering (FFR), Gaze-Tracked Foveated Rendering (TFR), and conventional, non-foveated rendering. While existing papers have often studied these methods, we provide the first comprehensive study of their relative feasibility in practical systems with limited battery life and computational budget. We show that TFR with the added cost of the gaze-tracker can often be more expensive than FFR. Thus, we co-design a gaze-tracked foveated renderer considering its benefits in computation, power efficiency, and tradeoffs in image quality. We describe principled approximations for eye tracking which provide up to a 9x speedup in runtime performance with approximately a 20x improvement in energy efficiency when run on a mobile GPU. In isolation, these approximations appear to significantly degrade the gaze quality, but appropriate compensation in the visual pipeline can mitigate the loss. Overall, we show that with a highly optimized gaze-tracker, TFR is feasible compared to FFR, resulting in up to 1.25x faster frame times while also reducing total energy consumption by over 40%. |
|---|---|
| AbstractList | Extended reality (XR) devices, including augmented, virtual, and mixed reality, provide a deeply immersive experience. However, practical limitations like weight, heat, and comfort put extreme constraints on the performance, power consumption, and image quality of such systems. In this paper, we study how these constraints form the tradeoff between Fixed Foveated Rendering (FFR), Gaze-Tracked Foveated Rendering (TFR), and conventional, non-foveated rendering. While existing papers have often studied these methods, we provide the first comprehensive study of their relative feasibility in practical systems with limited battery life and computational budget. We show that TFR with the added cost of the gaze-tracker can often be more expensive than FFR. Thus, we co-design a gaze-tracked foveated renderer considering its benefits in computation, power efficiency, and tradeoffs in image quality. We describe principled approximations for eye tracking which provide up to a 9x speedup in runtime performance with approximately a 20x improvement in energy efficiency when run on a mobile GPU. In isolation, these approximations appear to significantly degrade the gaze quality, but appropriate compensation in the visual pipeline can mitigate the loss. Overall, we show that with a highly optimized gaze-tracker, TFR is feasible compared to FFR, resulting in up to 1.25x faster frame times while also reducing total energy consumption by over 40%. |
| Author | Zhao, Yifan Huzaifa, Muhammad Patney, Anjul Singh, Rahul Liu, Jeffrey Sharif, Hashim Adve, Sarita |
| Author_xml | – sequence: 1 givenname: Rahul surname: Singh fullname: Singh, Rahul email: rahuls10@illinois.edu organization: University of Illinois Urbana-Champaign – sequence: 2 givenname: Muhammad surname: Huzaifa fullname: Huzaifa, Muhammad email: huzaifa2@illinois.edu organization: University of Illinois Urbana-Champaign – sequence: 3 givenname: Jeffrey surname: Liu fullname: Liu, Jeffrey email: jliu179@illinois.edu organization: University of Illinois Urbana-Champaign – sequence: 4 givenname: Anjul surname: Patney fullname: Patney, Anjul email: apatney@nvidia.com organization: NVIDIA – sequence: 5 givenname: Hashim surname: Sharif fullname: Sharif, Hashim email: hsharif3@illinois.edu organization: University of Illinois Urbana-Champaign – sequence: 6 givenname: Yifan surname: Zhao fullname: Zhao, Yifan email: yifanz16@illinois.edu organization: University of Illinois Urbana-Champaign – sequence: 7 givenname: Sarita surname: Adve fullname: Adve, Sarita email: sadve@illinois.edu organization: University of Illinois Urbana-Champaign |
| BookMark | eNotjc1KAzEURqMoWOu8gLjIA3TqzX-ylNJqoWAt1W3JTG7KSCcjmVHp21tRvsXZHM53TS5Sl5CQWwZTxsDdv22UYkpOOXAxBQChz0jhjLNCgZCWKX1ORlxLXiqu5BUp-v79V-MgTxuR-br7xjyha8yxy61PNU6oT4EuW79H-vLpD81wpNvsA3Yx9rRJdNF9oR8w0A2mgLlJ-xtyGf2hx-KfY_K6mG9nT-Xq-XE5e1iVzelvKGM0QVcGGTe1CdKAjcIED1Fa76V22olKozbGaCZFjcazqBxjFUd0TtZiTO7-ug0i7j5y0_p83DFgYMFK8QPurkz9 |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/VR55154.2023.00036 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9798350348156 |
| EISSN | 2642-5254 |
| EndPage | 214 |
| ExternalDocumentID | 10108084 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IF 6IH 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-i204t-ff7d6b7e127c7d4708f37da0f48aa469693b6e67776143ce7a1f5911b2ee994c3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 23 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000990497900021&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:47:27 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i204t-ff7d6b7e127c7d4708f37da0f48aa469693b6e67776143ce7a1f5911b2ee994c3 |
| PageCount | 10 |
| ParticipantIDs | ieee_primary_10108084 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-March |
| PublicationDateYYYYMMDD | 2023-03-01 |
| PublicationDate_xml | – month: 03 year: 2023 text: 2023-March |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings (IEEE Conference on Virtual Reality and 3D User Interfaces. Online) |
| PublicationTitleAbbrev | VR |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003204040 |
| Score | 2.3442917 |
| Snippet | Extended reality (XR) devices, including augmented, virtual, and mixed reality, provide a deeply immersive experience. However, practical limitations like... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 205 |
| SubjectTerms | Human-centered computing-Visualization-Visualization design and evaluation methods Human-centered computing-Visualization-Visualization techniques-Treemaps Image quality Performance evaluation Pipelines Power demand Runtime Three-dimensional displays Visualization |
| Title | Power, Performance, and Image Quality Tradeoffs in Foveated Rendering |
| URI | https://ieeexplore.ieee.org/document/10108084 |
| WOSCitedRecordID | wos000990497900021&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwED5BxcAEiCLe8sDYgJO4djKjVrBUUQWoW-XHWepAgvqS-PecnT5YGNiiDInubN_L990H8CA86nADlPRTwylBsT4xSpiEkhGXWqel5TqSTajRqJhMymoDVo9YGESMzWf4GB7jXb5r7CqUyuiEh464QhzCoVKyBWvtCip5RvtR8C0whpdPH2MKB2LhJAtjTHkYw_yLQiV6kOHJP_99Ct09Fo9VOy9zBgdYn8OgCuRmPVbt2_57TNeOvX6SfWDtYIxvRo7IYeP9gs1qNmzWZHfRsXFkj6OPdeF9OHh7fkk2hAjJjIRbJt4rJ43CNFNWOaF44XPlNPei0JryXFnmRqJUpBwKgywqnfo-WTOTIZalsPkFdOqmxktgtkAtMQ0sv4ZWyBpf9J3JHIVzFECU2RV0gw6mX-3Mi-lW_Os_3t_AcVBz2511C53lfIV3cGTXy9lifh9X6gcIFZPK |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV05T8MwFLagIMEEiCJuPDA24CROnMyoVStKFVUFdat8PEsdSFAviX_Ps9ODhYHN8mDL17v8vvcR8sgtSPcDFCShYuigaBsowVWAzogJtZGpZtKTTYjBIBuP82INVvdYGADwyWfw5Jr-L99UeulCZfjCXUZcxvfJgaPOYjVcaxtSiSO8kZxtoDEsf_4YokHgQyeRK2TKXCHmXyQqXod0Tv45-ylp7tB4tNjqmTOyB-U5aReO3qxFi13if4vK0tDeJ0oIWpfG-KaoigxU1s7ptKSdaoWSFwwdev44HKxJ3jvt0Us3WFMiBFNc3CKwVphUCQgjoYXhgmU2FkYyyzMp0dNN81ilkAohUO3GGoQMbYLyTEUAec51fEEaZVXCJaE6A5lC6Hh-FZ6RVjZLjIoMGnRoQuTRFWm6PZh81VUvJpvlX__R_0COuqO3_qTfG7zekGO35XWu1i1pLGZLuCOHerWYzmf3_tR-AOomlxA |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Conference+on+Virtual+Reality+and+3D+User+Interfaces.+Online%29&rft.atitle=Power%2C+Performance%2C+and+Image+Quality+Tradeoffs+in+Foveated+Rendering&rft.au=Singh%2C+Rahul&rft.au=Huzaifa%2C+Muhammad&rft.au=Liu%2C+Jeffrey&rft.au=Patney%2C+Anjul&rft.date=2023-03-01&rft.pub=IEEE&rft.eissn=2642-5254&rft.spage=205&rft.epage=214&rft_id=info:doi/10.1109%2FVR55154.2023.00036&rft.externalDocID=10108084 |