Perceptual color transformations for wide color gamut video coding
Gespeichert in:
| Titel: | Perceptual color transformations for wide color gamut video coding |
|---|---|
| Patent Number: | 10257,526 |
| Publikationsdatum: | April 09, 2019 |
| Appl. No: | 14/862027 |
| Application Filed: | September 22, 2015 |
| Abstract: | A video coding pipeline is provided that can accommodate high dynamic range (HDR) and wide color gamut (WCG) content at a fixed bitrate. The video coding pipeline relies on separate chromaticity and luminance-specific transforms in order to process image content. Image content may be converted into a nearly perceptually uniform color space for coding in constant luminance. Moreover, chromaticity transforms are utilized which reduce coding errors in the chroma components (at the fixed bitrate) by enlarging the distribution of code words for compression. |
| Inventors: | Disney Enterprises, Inc. (Burbank, CA, US); Eidgenoessische Technische Hochschule Zurich (ETH Zurich) (Zurich, CH) |
| Assignees: | Disney Enterprises, Inc. (Burbank, CA, US) |
| Claim: | 1. A computer-implemented method, comprising: converting an additive color model signal into a uniform color space signal having chromaticity components and a luminance component; transforming the chromaticity components by applying a chromaticity-specific transform to the chromaticity components, wherein the chromaticity-specific transform maps a main triangular representation of the chromaticity components in the uniform color space to a four-sided polygonal representation of the chromaticity components in the uniform color space; transforming the luminance component by applying a luminance-specific transform to the luminance component; quantizing the transformed chromaticity components and luminance component; and encoding the quantized chromaticity components and luminance component. |
| Claim: | 2. The computer-implemented method of claim 1 , wherein the additive color model signal comprises a red-green-blue (RGB) color model signal. |
| Claim: | 3. The computer-implemented method of claim 1 , wherein the uniform color space signal comprises a YUV color space signal. |
| Claim: | 4. The computer-implemented method of claim 3 , wherein mapping a main triangular representation of the chromaticity components in the uniform color space comprises subdividing the main triangular representation of the chromaticity components in the YUV color space into a mesh comprising a plurality of secondary triangles. |
| Claim: | 5. The computer-implemented method of claim 4 , wherein each of the vertices of the main triangular representation comprises one of the primaries of the additive color model. |
| Claim: | 6. The computer-implemented method of claim 5 , further comprising defining a midway vertex between two of the primaries of the additive color model. |
| Claim: | 7. The computer-implemented method of claim 6 , further comprising mapping each of the vertices of the main triangular representation and the defined midway vertex to a rectangular representation of the chromaticity components in the YUV color space. |
| Claim: | 8. The computer-implemented method of claim 7 , wherein the rectangular representation comprises a unit square. |
| Claim: | 9. The computer-implemented method of claim 7 , wherein respective vertices of each of the plurality of secondary triangles is mapped to the rectangular representation of the chromaticity components in the YUV color space using mean value coordinates. |
| Claim: | 10. The computer-implemented method of claim 9 , wherein values of each of the chromaticity components in the YUV color space are mapped to the rectangular representation thereof using barycentric coordinates corresponding to the values of each of the chromaticity components. |
| Claim: | 11. The computer-implemented method of claim 6 , further comprising mapping each of the vertices of the main triangular representation and the defined midway vertex to a four-sided polygonal representation of the chromaticity components in the YUV color space. |
| Claim: | 12. The computer-implemented method of claim 11 , further comprising applying at least one of a plurality of constraints affecting the mapping of each of the vertices of the main triangular representation, each of the vertices of each of the plurality of secondary triangles, and the defined midway vertex to the four-sided polygonal representation. |
| Claim: | 13. The computer-implemented method of claim 12 , further comprising adjusting characteristics of at least one of the plurality of constraints to modify the chromaticity-specific transform. |
| Claim: | 14. The computer-implemented method of claim 1 , wherein the luminance-specific transform comprises an opto-electrical transfer function. |
| Claim: | 15. A non-transitory computer readable medium having computer executable program code embodied thereon, the computer executable program code configured to cause a processor to: decode quantized color and brightness components of an input bitstream carrying an image signal represented by a first color space model having brightness and color components; dequantize the quantized color and brightness components; apply a first inverse transform to the dequantized color component, wherein the first inverse transform maps a four-sided polygonal representation of the color component in the first color space model to a triangular representation of the color components in the first color space model; apply a second inverse transform to the dequantized brightness component separately from applying the first inverse transform to the color component; and convert the image signal represented by a first color space model into that represented by a second color space model, the second color space model being an additive color space model. |
| Claim: | 16. The non-transitory computer readable medium of claim 15 , wherein the first inverse transform comprises one of an inverse direct transform or an inverse least squares transform, wherein the color and brightness components have been previously transformed to increase the number of code words available to a video codec for compressing the image signal. |
| Claim: | 17. The non-transitory computer readable medium of claim 16 , wherein the first color space model comprises a YUV color space model, the second color space model comprises a red-green-blue (RGB) color space model, and the video codec comprises a high efficiency video coding (HEVC) codec. |
| Claim: | 18. The non-transitory computer readable medium of claim 15 , wherein the image signal comprises one of a standard dynamic range (SDR) image signal, a high dynamic range (HDR) image signal, a standard color gamut (SCG) image signal, or a wide color gamut (WCG) image signal. |
| Claim: | 19. A system, comprising: a processor comprising: a converter, a first transformer, an opto-electrical transfer function, and an encoder; a non-transitory computer-readable medium coupled to the converter, the first transformer, the opto-electrical transfer function, and the encoder, the non-transitory computer-readable medium having computer executable program code embodied thereon, the computer executable program code configured to cause a processor to: convert an image signal represented by a first color space model into that represented by a second color space model having brightness and color components using the converter; apply a first transform to the color component using the first transformer, wherein the first transformer maps a triangular representation of the color components in the second color space model to a four-sided polygonal representation of the color components in the second color space model; apply a second transform to the brightness component using the encoder separately from applying the first transform to the color component; quantize the transformed color and brightness components using the quantizer; and encode the quantized color and brightness components and output a bitstream using the encoder. |
| Claim: | 20. The system of claim 19 , wherein the first transform is invertible. |
| Patent References Cited: | 6100998 August 2000 Nagao 8783901 July 2014 Zoorob 9584811 February 2017 Su 2004/0096104 May 2004 Terekhov 2005/0184998 August 2005 Yang 2007/0086027 April 2007 Tin 2008/0259400 October 2008 Hersch 2009/0285475 November 2009 Suzuki 2010/0060911 March 2010 Marcu 2010/0086226 April 2010 Rozzi 2014/0105277 April 2014 Bivolarsky 2015/0103902 April 2015 Li 2015/0116740 April 2015 Miyanaga 2016/0309154 October 2016 Rusanovskyy 2016/0366449 December 2016 Stessen 2018/0007372 January 2018 Lasserre 1875398 December 2006 103563376 February 2014 103827956 May 2014 104486605 April 2015 |
| Other References: | Farbman et al., Coordinates for instant image cloning, ACM Trans. Graph., Jul. 2009, pp. 1-9, 28(3):67. cited by applicant Ford, Adrian, Colour space conversions, Aug. 1998, pp. 1-31. cited by applicant Klaus,Simon, Farbe im Digitalen Publizieren, Springer, 2008, English abstract translation. cited by applicant Luthra et al., Draft requirements and explorations for hdr and wcg content distribution, International Organisation for Standardisation, Apr. 2014, pp. 1-22, Valencia, Spain. cited by applicant Luthra et al., Test sequences and anchor generation for hdr and wide gamut content distribution, International Organisation for Standardisation, Jul. 2014, pp. 1-15, Sapporo, Japan. cited by applicant Pedzisa, Maciej Pedzisz, Beyond bt.709, SMPTE Motion Imaging Journal, 2014, pp. 1-8, 123:18-25. cited by applicant Schubert, E. Fred , Chapter 17: Colorimetry, http://www.ecse.rpi.edu/˜schubert/Light-Emitting-Diodes-dot-org/chap17/chap17.htm, last accessed Mar. 21, 2016, pp. 1-15. cited by applicant Sullivan, et al., Overview of the high efficiency video coding (hevc) standard, IEEE Transactions on Circuits and Systems for Video Technology, Dec. 12, 2012, pp. 1649-1668, 22(12). cited by applicant First Office Action in Chinese Patent Application No. 2016102814925, dated Aug. 3, 2018. cited by applicant |
| Assistant Examiner: | Sullivan, Tyler W. |
| Primary Examiner: | Czekaj, Dave |
| Attorney, Agent or Firm: | Sheppard Mullin Richter & Hampton LLP |
| Dokumentencode: | edspgr.10257526 |
| Datenbank: | USPTO Patent Grants |
| FullText | Text: Availability: 0 CustomLinks: – Url: https://ppubs.uspto.gov/pubwebapp/external.html?q=(%2210257%2C526%22).pn.&db=USPAT&type=ids Name: EDS - USPTO Patent Grants (s4221598) Category: fullText Text: View record in USPTO Patent Grants |
|---|---|
| Header | DbId: edspgr DbLabel: USPTO Patent Grants An: edspgr.10257526 RelevancyScore: 779 AccessLevel: 3 PubType: Patent PubTypeId: patent PreciseRelevancyScore: 779.293762207031 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Perceptual color transformations for wide color gamut video coding – Name: DocumentID Label: Patent Number Group: Patent Data: 10257,526 – Name: DateEntry Label: Publication Date Group: Patent Data: April 09, 2019 – Name: DocumentID Label: Appl. No Group: Patent Data: 14/862027 – Name: DateFiled Label: Application Filed Group: Patent Data: September 22, 2015 – Name: Abstract Label: Abstract Group: Ab Data: A video coding pipeline is provided that can accommodate high dynamic range (HDR) and wide color gamut (WCG) content at a fixed bitrate. The video coding pipeline relies on separate chromaticity and luminance-specific transforms in order to process image content. Image content may be converted into a nearly perceptually uniform color space for coding in constant luminance. Moreover, chromaticity transforms are utilized which reduce coding errors in the chroma components (at the fixed bitrate) by enlarging the distribution of code words for compression. – Name: Author Label: Inventors Group: Patent Data: <searchLink fieldCode="ZA" term="%22Disney+Enterprises%2C+Inc%2E%22">Disney Enterprises, Inc.</searchLink> (Burbank, CA, US); <searchLink fieldCode="ZA" term="%22Eidgenoessische+Technische+Hochschule+Zurich+%28ETH+Zurich%29%22">Eidgenoessische Technische Hochschule Zurich (ETH Zurich)</searchLink> (Zurich, CH) – Name: OtherAuthors Label: Assignees Group: Patent Data: <searchLink fieldCode="ZS" term="%22Disney+Enterprises%2C+Inc%2E%22">Disney Enterprises, Inc.</searchLink> (Burbank, CA, US) – Name: Comment Label: Claim Group: Patent Data: 1. A computer-implemented method, comprising: converting an additive color model signal into a uniform color space signal having chromaticity components and a luminance component; transforming the chromaticity components by applying a chromaticity-specific transform to the chromaticity components, wherein the chromaticity-specific transform maps a main triangular representation of the chromaticity components in the uniform color space to a four-sided polygonal representation of the chromaticity components in the uniform color space; transforming the luminance component by applying a luminance-specific transform to the luminance component; quantizing the transformed chromaticity components and luminance component; and encoding the quantized chromaticity components and luminance component. – Name: Comment Label: Claim Group: Patent Data: 2. The computer-implemented method of claim 1 , wherein the additive color model signal comprises a red-green-blue (RGB) color model signal. – Name: Comment Label: Claim Group: Patent Data: 3. The computer-implemented method of claim 1 , wherein the uniform color space signal comprises a YUV color space signal. – Name: Comment Label: Claim Group: Patent Data: 4. The computer-implemented method of claim 3 , wherein mapping a main triangular representation of the chromaticity components in the uniform color space comprises subdividing the main triangular representation of the chromaticity components in the YUV color space into a mesh comprising a plurality of secondary triangles. – Name: Comment Label: Claim Group: Patent Data: 5. The computer-implemented method of claim 4 , wherein each of the vertices of the main triangular representation comprises one of the primaries of the additive color model. – Name: Comment Label: Claim Group: Patent Data: 6. The computer-implemented method of claim 5 , further comprising defining a midway vertex between two of the primaries of the additive color model. – Name: Comment Label: Claim Group: Patent Data: 7. The computer-implemented method of claim 6 , further comprising mapping each of the vertices of the main triangular representation and the defined midway vertex to a rectangular representation of the chromaticity components in the YUV color space. – Name: Comment Label: Claim Group: Patent Data: 8. The computer-implemented method of claim 7 , wherein the rectangular representation comprises a unit square. – Name: Comment Label: Claim Group: Patent Data: 9. The computer-implemented method of claim 7 , wherein respective vertices of each of the plurality of secondary triangles is mapped to the rectangular representation of the chromaticity components in the YUV color space using mean value coordinates. – Name: Comment Label: Claim Group: Patent Data: 10. The computer-implemented method of claim 9 , wherein values of each of the chromaticity components in the YUV color space are mapped to the rectangular representation thereof using barycentric coordinates corresponding to the values of each of the chromaticity components. – Name: Comment Label: Claim Group: Patent Data: 11. The computer-implemented method of claim 6 , further comprising mapping each of the vertices of the main triangular representation and the defined midway vertex to a four-sided polygonal representation of the chromaticity components in the YUV color space. – Name: Comment Label: Claim Group: Patent Data: 12. The computer-implemented method of claim 11 , further comprising applying at least one of a plurality of constraints affecting the mapping of each of the vertices of the main triangular representation, each of the vertices of each of the plurality of secondary triangles, and the defined midway vertex to the four-sided polygonal representation. – Name: Comment Label: Claim Group: Patent Data: 13. The computer-implemented method of claim 12 , further comprising adjusting characteristics of at least one of the plurality of constraints to modify the chromaticity-specific transform. – Name: Comment Label: Claim Group: Patent Data: 14. The computer-implemented method of claim 1 , wherein the luminance-specific transform comprises an opto-electrical transfer function. – Name: Comment Label: Claim Group: Patent Data: 15. A non-transitory computer readable medium having computer executable program code embodied thereon, the computer executable program code configured to cause a processor to: decode quantized color and brightness components of an input bitstream carrying an image signal represented by a first color space model having brightness and color components; dequantize the quantized color and brightness components; apply a first inverse transform to the dequantized color component, wherein the first inverse transform maps a four-sided polygonal representation of the color component in the first color space model to a triangular representation of the color components in the first color space model; apply a second inverse transform to the dequantized brightness component separately from applying the first inverse transform to the color component; and convert the image signal represented by a first color space model into that represented by a second color space model, the second color space model being an additive color space model. – Name: Comment Label: Claim Group: Patent Data: 16. The non-transitory computer readable medium of claim 15 , wherein the first inverse transform comprises one of an inverse direct transform or an inverse least squares transform, wherein the color and brightness components have been previously transformed to increase the number of code words available to a video codec for compressing the image signal. – Name: Comment Label: Claim Group: Patent Data: 17. The non-transitory computer readable medium of claim 16 , wherein the first color space model comprises a YUV color space model, the second color space model comprises a red-green-blue (RGB) color space model, and the video codec comprises a high efficiency video coding (HEVC) codec. – Name: Comment Label: Claim Group: Patent Data: 18. The non-transitory computer readable medium of claim 15 , wherein the image signal comprises one of a standard dynamic range (SDR) image signal, a high dynamic range (HDR) image signal, a standard color gamut (SCG) image signal, or a wide color gamut (WCG) image signal. – Name: Comment Label: Claim Group: Patent Data: 19. A system, comprising: a processor comprising: a converter, a first transformer, an opto-electrical transfer function, and an encoder; a non-transitory computer-readable medium coupled to the converter, the first transformer, the opto-electrical transfer function, and the encoder, the non-transitory computer-readable medium having computer executable program code embodied thereon, the computer executable program code configured to cause a processor to: convert an image signal represented by a first color space model into that represented by a second color space model having brightness and color components using the converter; apply a first transform to the color component using the first transformer, wherein the first transformer maps a triangular representation of the color components in the second color space model to a four-sided polygonal representation of the color components in the second color space model; apply a second transform to the brightness component using the encoder separately from applying the first transform to the color component; quantize the transformed color and brightness components using the quantizer; and encode the quantized color and brightness components and output a bitstream using the encoder. – Name: Comment Label: Claim Group: Patent Data: 20. The system of claim 19 , wherein the first transform is invertible. – Name: Ref Label: Patent References Cited Group: Patent Data: <searchLink fieldCode="RF" term="%226100998%22">6100998</searchLink> August 2000 Nagao<br /><searchLink fieldCode="RF" term="%228783901%22">8783901</searchLink> July 2014 Zoorob<br /><searchLink fieldCode="RF" term="%229584811%22">9584811</searchLink> February 2017 Su<br /><searchLink fieldCode="RF" term="%222004%2E0096104%22">2004/0096104</searchLink> May 2004 Terekhov<br /><searchLink fieldCode="RF" term="%222005%2E0184998%22">2005/0184998</searchLink> August 2005 Yang<br /><searchLink fieldCode="RF" term="%222007%2E0086027%22">2007/0086027</searchLink> April 2007 Tin<br /><searchLink fieldCode="RF" term="%222008%2E0259400%22">2008/0259400</searchLink> October 2008 Hersch<br /><searchLink fieldCode="RF" term="%222009%2E0285475%22">2009/0285475</searchLink> November 2009 Suzuki<br /><searchLink fieldCode="RF" term="%222010%2E0060911%22">2010/0060911</searchLink> March 2010 Marcu<br /><searchLink fieldCode="RF" term="%222010%2E0086226%22">2010/0086226</searchLink> April 2010 Rozzi<br /><searchLink fieldCode="RF" term="%222014%2E0105277%22">2014/0105277</searchLink> April 2014 Bivolarsky<br /><searchLink fieldCode="RF" term="%222015%2E0103902%22">2015/0103902</searchLink> April 2015 Li<br /><searchLink fieldCode="RF" term="%222015%2E0116740%22">2015/0116740</searchLink> April 2015 Miyanaga<br /><searchLink fieldCode="RF" term="%222016%2E0309154%22">2016/0309154</searchLink> October 2016 Rusanovskyy<br /><searchLink fieldCode="RF" term="%222016%2E0366449%22">2016/0366449</searchLink> December 2016 Stessen<br /><searchLink fieldCode="RF" term="%222018%2E0007372%22">2018/0007372</searchLink> January 2018 Lasserre<br /><searchLink fieldCode="RF" term="%221875398%22">1875398</searchLink> December 2006<br /><searchLink fieldCode="RF" term="%22103563376%22">103563376</searchLink> February 2014<br /><searchLink fieldCode="RF" term="%22103827956%22">103827956</searchLink> May 2014<br /><searchLink fieldCode="RF" term="%22104486605%22">104486605</searchLink> April 2015<br /><br /><br /><br /><br /><br /><br /><br /><br /> – Name: Ref Label: Other References Group: Patent Data: Farbman et al., Coordinates for instant image cloning, ACM Trans. Graph., Jul. 2009, pp. 1-9, 28(3):67. cited by applicant<br />Ford, Adrian, Colour space conversions, Aug. 1998, pp. 1-31. cited by applicant<br />Klaus,Simon, Farbe im Digitalen Publizieren, Springer, 2008, English abstract translation. cited by applicant<br />Luthra et al., Draft requirements and explorations for hdr and wcg content distribution, International Organisation for Standardisation, Apr. 2014, pp. 1-22, Valencia, Spain. cited by applicant<br />Luthra et al., Test sequences and anchor generation for hdr and wide gamut content distribution, International Organisation for Standardisation, Jul. 2014, pp. 1-15, Sapporo, Japan. cited by applicant<br />Pedzisa, Maciej Pedzisz, Beyond bt.709, SMPTE Motion Imaging Journal, 2014, pp. 1-8, 123:18-25. cited by applicant<br />Schubert, E. Fred , Chapter 17: Colorimetry, http://www.ecse.rpi.edu/˜schubert/Light-Emitting-Diodes-dot-org/chap17/chap17.htm, last accessed Mar. 21, 2016, pp. 1-15. cited by applicant<br />Sullivan, et al., Overview of the high efficiency video coding (hevc) standard, IEEE Transactions on Circuits and Systems for Video Technology, Dec. 12, 2012, pp. 1649-1668, 22(12). cited by applicant<br />First Office Action in Chinese Patent Application No. 2016102814925, dated Aug. 3, 2018. cited by applicant – Name: AuthorEditor Label: Assistant Examiner Group: Patent Data: <searchLink fieldCode="ZE" term="%22Sullivan%2C+Tyler+W%2E%22">Sullivan, Tyler W.</searchLink> – Name: AuthorEditor Label: Primary Examiner Group: Patent Data: <searchLink fieldCode="ZE" term="%22Czekaj%2C+Dave%22">Czekaj, Dave</searchLink> – Name: AuthorCorporate Label: Attorney, Agent or Firm Group: Patent Data: <searchLink fieldCode="ZG" term="%22Sheppard+Mullin+Richter+%26+Hampton+LLP%22">Sheppard Mullin Richter & Hampton LLP</searchLink> – Name: AN Label: Accession Number Group: ID Data: edspgr.10257526 |
| PLink | https://erproxy.cvtisr.sk/sfx/access?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edspgr&AN=edspgr.10257526 |
| RecordInfo | BibRecord: BibEntity: Languages: – Text: English Titles: – TitleFull: Perceptual color transformations for wide color gamut video coding Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Disney Enterprises, Inc. – PersonEntity: Name: NameFull: Eidgenoessische Technische Hochschule Zurich (ETH Zurich) IsPartOfRelationships: – BibEntity: Dates: – D: 09 M: 04 Text: April 09, 2019 Type: published Y: 2019 |
| ResultId | 1 |