Partition-Aware Adaptive Switching Neural Networks for Post-Processing in HEVC

This article addresses neural network based post-processing for the state-of-the-art video coding standard, High Efficiency Video Coding (HEVC). We first propose a partition-aware convolution neural network (CNN) that utilizes the partition information produced by the encoder to assist in the post-p...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on multimedia Vol. 22; no. 11; pp. 2749 - 2763
Main Authors: Lin, Weiyao, He, Xiaoyi, Han, Xintong, Liu, Dong, See, John, Zou, Junni, Xiong, Hongkai, Wu, Feng
Format: Journal Article
Language:English
Published: Piscataway IEEE 01.11.2020
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Artificial neural networks
Coders
Coding standards
Convolution
convolutional neural network
Degradation
Encoding
High Efficiency Video Coding
Image coding
Neural networks
Partitions
Post-processing
Switching
Training
Video coding
Video compression
Visualization
ISSN:1520-9210, 1941-0077
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This article addresses neural network based post-processing for the state-of-the-art video coding standard, High Efficiency Video Coding (HEVC). We first propose a partition-aware convolution neural network (CNN) that utilizes the partition information produced by the encoder to assist in the post-processing. In contrast to existing CNN-based approaches, which only take the decoded frame as input, the proposed approach considers the coding unit (CU) size information and combines it with the distorted decoded frame such that the artifacts introduced by HEVC are efficiently reduced. We further introduce an adaptive-switching neural network (ASN) that consists of multiple independent CNNs to adaptively handle the variations in content and distortion within compressed-video frames, providing further reduction in visual artifacts. Additionally, an iterative training procedure is proposed to train these independent CNNs attentively on different local patch-wise classes. Experiments on benchmark sequences demonstrate the effectiveness of our partition-aware and adaptive-switching neural networks.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2019.2962310