IQNet: Image Quality Assessment Guided Just Noticeable Difference Prefiltering for Versatile Video Coding

Image prefiltering with just noticeable distortion (JND) improves coding efficiency in a visual lossless way by filtering the perceptually redundant information prior to compression. However, real JND cannot be well modeled with inaccurate masking equations in traditional approaches or image-level s...

Full description

Saved in:
Bibliographic Details
Published in:IEEE open journal of circuits and systems Vol. 5; pp. 17 - 27
Main Authors: Sun, Yu-Han, Lee, Chiang Lo-Hsuan, Chang, Tian-Sheuan
Format: Journal Article
Language:English
Published: New York IEEE 2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Codecs
Datasets
Deep learning
Discrete cosine transforms
Distortion
Encoding
Image coding
Image edge detection
Image enhancement
Image quality
Image reconstruction
just noticeable distortion
Quality assessment
Training data
Video coding
video quality assessment
ISSN:2644-1225, 2644-1225
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Image prefiltering with just noticeable distortion (JND) improves coding efficiency in a visual lossless way by filtering the perceptually redundant information prior to compression. However, real JND cannot be well modeled with inaccurate masking equations in traditional approaches or image-level subject tests in deep learning approaches. Thus, this paper proposes a fine-grained JND prefiltering dataset guided by image quality assessment for accurate block-level JND modeling. The dataset is constructed from decoded images to include coding effects and is also perceptually enhanced with block overlap and edge preservation. Furthermore, based on this dataset, we propose a lightweight JND prefiltering network, IQNet, which can be applied directly to different quantization cases with the same model and only needs 3K parameters. The experimental results show that the proposed approach to Versatile Video Coding could yield maximum/average bitrate savings of 41%/15% and 53%/19% for all-intra and low-delay P configurations, respectively, with negligible subjective quality loss. Our method demonstrates higher perceptual quality and a model size that is an order of magnitude smaller than previous deep learning methods.
AbstractList Image prefiltering with just noticeable distortion (JND) improves coding efficiency in a visual lossless way by filtering the perceptually redundant information prior to compression. However, real JND cannot be well modeled with inaccurate masking equations in traditional approaches or image-level subject tests in deep learning approaches. Thus, this paper proposes a fine-grained JND prefiltering dataset guided by image quality assessment for accurate block-level JND modeling. The dataset is constructed from decoded images to include coding effects and is also perceptually enhanced with block overlap and edge preservation. Furthermore, based on this dataset, we propose a lightweight JND prefiltering network, IQNet, which can be applied directly to different quantization cases with the same model and only needs 3K parameters. The experimental results show that the proposed approach to Versatile Video Coding could yield maximum/average bitrate savings of 41%/15% and 53%/19% for all-intra and low-delay P configurations, respectively, with negligible subjective quality loss. Our method demonstrates higher perceptual quality and a model size that is an order of magnitude smaller than previous deep learning methods.
Author Lee, Chiang Lo-Hsuan
Chang, Tian-Sheuan
Sun, Yu-Han
Author_xml – sequence: 1
  givenname: Yu-Han
  surname: Sun
  fullname: Sun, Yu-Han
  organization: Institute of Electronics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
– sequence: 2
  givenname: Chiang Lo-Hsuan
  surname: Lee
  fullname: Lee, Chiang Lo-Hsuan
  organization: Institute of Electronics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
– sequence: 3
  givenname: Tian-Sheuan
  orcidid: 0000-0002-0561-8745
  surname: Chang
  fullname: Chang, Tian-Sheuan
  email: tschang@nycu.edu.tw
  organization: Institute of Electronics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
BookMark eNpNUU1PGzEQtSoqlVL-AOJgqeek_t5dblFKaRCCohau1ng9jhxt1tTePfDvawiqOM1o5r03H-8zORrTiISccbbknHXf7q7Xq99LwYRcSqkU69QHciyMUgsuhD56l38ip6XsGGNCc85Fc0zi5v4Wpwu62cMW6f0MQ5ye6aoULGWP40Sv5ujR0-u5TPQ2TbFHcAPS7zEEzDj2SH9lDHGYMMdxS0PK9BFzgSlW1GPlJrpOvra-kI8BhoKnb_GEPPy4_LP-ubi5u9qsVzeLXupmWnRac6FZaIPooG-FAQOeg_ZCuaBBC_SOISofJIO26blxzjW9EegajibIE7I56PoEO_uU4x7ys00Q7Wsh5a2FXO8Y0OpWhhawV61SykkA07W-dSiUbhjTrmp9PWg95fR3xjLZXZrzWNe3ohOSSyOZrihxQPU5lVK_8X8qZ_bFIfvqkH1xyL45VEnnB1JExHcEabRmnfwHTaWOeQ
CODEN IOJCC3
Cites_doi 10.1109/ICIP.2019.8803454
10.1016/j.jvcir.2017.04.009
10.1016/j.image.2023.117019
10.1109/TCSVT.2020.3030895
10.1109/TCSVT.2021.3076224
10.1016/j.ins.2023.119575
10.1109/CVPR.2012.6247954
10.1007/s11263-018-01144-2
10.1109/TCSVT.2016.2539862
10.1007/978-3-030-67070-2_3
10.1109/TCSVT.2015.2389491
10.1109/ICIP49359.2023.10222099
10.1109/TCSVT.2021.3101953
10.1109/ISCAS.2016.7539109
10.1109/ICIP.2016.7532610
10.1109/TIP.2018.2818439
10.1109/ICMEW46912.2020.9105999
10.1109/ACCESS.2020.3046194
10.1109/DCC50243.2021.00009
10.1109/CVPR46437.2021.01590
10.1109/TCSVT.2022.3163860
10.1016/j.jvcir.2023.103877
10.1109/TIP.2018.2831899
10.1109/ISPACS.2018.8923511
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID 97E
ESBDL
RIA
RIE
AAYXX
CITATION
7SP
8FD
L7M
DOA
DOI 10.1109/OJCAS.2023.3344094
DatabaseName IEEE Xplore (IEEE)
IEEE Xplore Open Access Journals
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
Electronics & Communications Abstracts
Technology Research Database
Advanced Technologies Database with Aerospace
DOAJ Directory of Open Access Journals
DatabaseTitle CrossRef
Technology Research Database
Advanced Technologies Database with Aerospace
Electronics & Communications Abstracts
DatabaseTitleList
Technology Research Database
Database_xml – sequence: 1
  dbid: DOA
  name: DOAJ Directory of Open Access Journals
  url: https://www.doaj.org/
  sourceTypes: Open Website
– sequence: 2
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 2644-1225
EndPage 27
ExternalDocumentID oai_doaj_org_article_583f8aec48444b3aa698d8be2457005b
10_1109_OJCAS_2023_3344094
10365509
Genre orig-research
GrantInformation_xml – fundername: National Science and Technology Council, Taiwan
  grantid: 111-2622-8-A49-018-SB; 110-2221-E-A49-148-MY3; 110-2218-E-A49-015-MBK
  funderid: 10.13039/501100020950
– fundername: Novatek
GroupedDBID 0R~
97E
AAJGR
ABAZT
ABVLG
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
EBS
ESBDL
GROUPED_DOAJ
JAVBF
M~E
OCL
OK1
RIA
RIE
AAYXX
CITATION
7SP
8FD
L7M
ID FETCH-LOGICAL-c357t-9551250f8f29ac826a6ad1a5d24bf5a52edb0ee4df30a87c16bbb7c62eb71e6f3
IEDL.DBID DOA
ISICitedReferencesCount 1
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001162376900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 2644-1225
IngestDate Fri Oct 03 12:44:25 EDT 2025
Sun Jun 29 15:58:13 EDT 2025
Sat Nov 29 05:59:26 EST 2025
Wed Aug 27 01:55:50 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Language English
License https://creativecommons.org/licenses/by-nc-nd/4.0
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c357t-9551250f8f29ac826a6ad1a5d24bf5a52edb0ee4df30a87c16bbb7c62eb71e6f3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0002-0561-8745
OpenAccessLink https://doaj.org/article/583f8aec48444b3aa698d8be2457005b
PQID 2923136305
PQPubID 5075790
PageCount 11
ParticipantIDs doaj_primary_oai_doaj_org_article_583f8aec48444b3aa698d8be2457005b
crossref_primary_10_1109_OJCAS_2023_3344094
ieee_primary_10365509
proquest_journals_2923136305
PublicationCentury 2000
PublicationDate 20240000
2024-00-00
20240101
2024-01-01
PublicationDateYYYYMMDD 2024-01-01
PublicationDate_xml – year: 2024
  text: 20240000
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE open journal of circuits and systems
PublicationTitleAbbrev OJCAS
PublicationYear 2024
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref12
ref15
ref14
ref11
ref10
ref2
ref17
ref16
ref19
ref18
Jinjian (ref1) 2018; 13
ref24
ref23
ref25
ref20
ref22
ref21
Li (ref26) 2016; 6
ref8
ref7
ref9
ref4
ref3
ref6
ref5
References_xml – ident: ref7
  doi: 10.1109/ICIP.2019.8803454
– ident: ref5
  doi: 10.1016/j.jvcir.2017.04.009
– ident: ref13
  doi: 10.1016/j.image.2023.117019
– ident: ref23
  doi: 10.1109/TCSVT.2020.3030895
– ident: ref8
  doi: 10.1109/TCSVT.2021.3076224
– ident: ref11
  doi: 10.1016/j.ins.2023.119575
– ident: ref22
  doi: 10.1109/CVPR.2012.6247954
– ident: ref25
  doi: 10.1007/s11263-018-01144-2
– ident: ref3
  doi: 10.1109/TCSVT.2016.2539862
– ident: ref24
  doi: 10.1007/978-3-030-67070-2_3
– ident: ref2
  doi: 10.1109/TCSVT.2015.2389491
– ident: ref10
  doi: 10.1109/ICIP49359.2023.10222099
– ident: ref18
  doi: 10.1109/TCSVT.2021.3101953
– ident: ref19
  doi: 10.1109/ISCAS.2016.7539109
– ident: ref6
  doi: 10.1109/ICIP.2016.7532610
– ident: ref4
  doi: 10.1109/TIP.2018.2818439
– ident: ref14
  doi: 10.1109/ICMEW46912.2020.9105999
– ident: ref17
  doi: 10.1109/ACCESS.2020.3046194
– volume: 6
  start-page: 2
  issue: 2
  year: 2016
  ident: ref26
  article-title: Toward a practical perceptual video quality metric
  publication-title: Netflix Tech. Blog
– ident: ref15
  doi: 10.1109/DCC50243.2021.00009
– ident: ref16
  doi: 10.1109/CVPR46437.2021.01590
– ident: ref9
  doi: 10.1109/TCSVT.2022.3163860
– ident: ref12
  doi: 10.1016/j.jvcir.2023.103877
– ident: ref21
  doi: 10.1109/TIP.2018.2831899
– volume: 13
  start-page: 4
  year: 2018
  ident: ref1
  article-title: Survey of visual just noticeable difference estimation
  publication-title: Front. Comput. Sci.
– ident: ref20
  doi: 10.1109/ISPACS.2018.8923511
SSID ssj0002511127
Score 2.2565181
Snippet Image prefiltering with just noticeable distortion (JND) improves coding efficiency in a visual lossless way by filtering the perceptually redundant...
SourceID doaj
proquest
crossref
ieee
SourceType Open Website
Aggregation Database
Index Database
Publisher
StartPage 17
SubjectTerms Codecs
Datasets
Deep learning
Discrete cosine transforms
Distortion
Encoding
Image coding
Image edge detection
Image enhancement
Image quality
Image reconstruction
just noticeable distortion
Quality assessment
Training data
Video coding
video quality assessment
SummonAdditionalLinks – databaseName: IEEE Electronic Library (IEL)
  dbid: RIE
  link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Lb9QwELbaqgc4AIUiFlrkAzeUJYnf3LZb-hLatuKh3iw_xtIe2KB2txL_nrGTXRUhDtyiyE4cfxnPN2PPDCHvVKwhaRkrxbSreBNUZZQwlUoiupoFVBulaslnNZvpmxtzNQSrl1gYACiHz2CcL8tefuzCKrvKUMKZREZttsm2UrIP1to4VDJXblq1DoypzYfLi-nkyzjXBx8zxrMh84fyKTn6h6Iqf63ERb2cPP3PgT0jTwYeSSc98HtkCxbPyeMH2QVfkPn59QyWH-n5D1wzaJ8s4xedbFJx0tPVPEKkuaAXnXX5FFyOo6LHQ82UAPQKP2Ge99PxgRTpLc3uNcQSW33Hvh2ddln37ZNvJ5--Ts-qobJCFZhQy8ogT0Luk3RqjQtoYTjpYuNEbLlPwokWoq8BeEysdlqFRnrvVZAteNWATOwl2Vl0C3hFaELrOgghUMclhJqh-RRr7RAOHV0MakTer6fc_uwTaNhieNTGFoBsBsgOAI3IUUZl0zInvy43cLrtIEtWaJa0g8A159wz56TRUXtoeU7WL_yI7GeIHryuR2dEDtYg20FS72ybGS6TuOy9_ke3N-QRDpH3fpcDsrO8XcEh2Q33y_nd7dvyE_4GOq7bBw
  priority: 102
  providerName: IEEE
Title IQNet: Image Quality Assessment Guided Just Noticeable Difference Prefiltering for Versatile Video Coding
URI https://ieeexplore.ieee.org/document/10365509
https://www.proquest.com/docview/2923136305
https://doaj.org/article/583f8aec48444b3aa698d8be2457005b
Volume 5
WOSCitedRecordID wos001162376900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVAON
  databaseName: DOAJ Directory of Open Access Journals
  customDbUrl:
  eissn: 2644-1225
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0002511127
  issn: 2644-1225
  databaseCode: DOA
  dateStart: 20200101
  isFulltext: true
  titleUrlDefault: https://www.doaj.org/
  providerName: Directory of Open Access Journals
– providerCode: PRVHPJ
  databaseName: ROAD: Directory of Open Access Scholarly Resources
  customDbUrl:
  eissn: 2644-1225
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0002511127
  issn: 2644-1225
  databaseCode: M~E
  dateStart: 20200101
  isFulltext: true
  titleUrlDefault: https://road.issn.org
  providerName: ISSN International Centre
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1Nb9MwGLZQtQMc0IAiysbkAzeUNont2ObWdRsUQSniQ71Z_pR6oJ3WdBKX_fa9b5JWnThw4ZJD5MTO-yTvh2M_DyFvZchjUlXIJFM244WXmZZCZzKJYHPmIWw0qiWf5WymFgs9P5D6wjVhLT1wa7iRUCwpGz1XnHPHrK20CsrFkiMzu3DofXOpD4op9MGYOBel3O2SyfXo66fJ-PsQxcKHjHGsah5Eooawv1NY-cstN7Hm6pg87ZJEOm4H94w8iqvn5MkBdeALspx-m8X6PZ3-BodAWyaMP3S859mkH7bLEANFtS46W-MSN9wkRS86QRQf6RyGtMSf5XBDCrkrxbkzAApa_YJr13SyxsDWJz-vLn9MPmadbELmmZB1piEJgsQmqVRq66F8sJUNhRWh5C4JK8oYXB4jD4nlVklfVM456asyOlnEKrGXpLdar-IrQhOUzl4IAQEsAY4MaqOQKwvmVcEGLwfk3c6E5rplxzBNVZFr0xjcoMFNZ_ABOUcr71sis3VzAvA2Hd7mX3gPSB8xOuiOVVBl6QE53YFmus9wY0pMX1kFPu31_-j7hDyG5-HtDMwp6dU32_iGHPnberm5OWveQDh-ubs8a_YR3gM9vt9h
linkProvider Directory of Open Access Journals
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Lb9QwEB7RglQ48GzFQgEfuKEsSWzHNrdloXRhCUUU1Jvlp7QHNqjdReLfM3ayqyLUQ29RZCeOv4znm7FnBuCl8GWIsvGFoNIUrHKiUIKrQkTuTUkdqo1ctWQu2laenamTIVg9x8KEEPLhszBOl3kv33dunVxlKOG0QUatduAmZ6wu-3CtrUslseWqFpvQmFK9_vJxOvk2ThXCx5SyZMr8o35ylv6hrMp_a3FWMEf3rjm0-3B3YJJk0kP_AG6E5UO4cym_4CNYzL62YfWGzH7iqkH6dBl_yGSbjJN8WC988CSV9CJtl87BpUgq8m6omuICOcFPWKQddXwgQYJLkoMN0cRWP7BvR6Zd0n778P3o_en0uBhqKxSOcrEqFDIlZD9RxloZhzaGaYyvDPc1s5EbXgdvyxCYj7Q0UriqsdYK19TBiio0kR7A7rJbhsdAItrXjnOOWi4i2BQNKF9Kg3BIb7wTI3i1mXL9q0-hobPpUSqdAdIJID0ANIK3CZVty5T-Ot_A6daDNGkuaZQmOCYZY5Ya0yjppQ01S-n6uR3BfoLo0ut6dEZwuAFZD7J6oevEcWmDC9-TK7q9gL3j089zPZ-1n57CbRwu670wh7C7Ol-HZ3DL_V4tLs6f5x_yL8QS3k4
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=IQNet%3A+Image+Quality+Assessment+Guided+Just+Noticeable+Difference+Prefiltering+for+Versatile+Video+Coding&rft.jtitle=IEEE+open+journal+of+circuits+and+systems&rft.au=Sun%2C+Yu-Han&rft.au=Lee%2C+Chiang+Lo-Hsuan&rft.au=Chang%2C+Tian-Sheuan&rft.date=2024&rft.pub=IEEE&rft.eissn=2644-1225&rft.volume=5&rft.spage=17&rft.epage=27&rft_id=info:doi/10.1109%2FOJCAS.2023.3344094&rft.externalDocID=10365509
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2644-1225&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2644-1225&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2644-1225&client=summon