PixelSieve: Towards Efficient Activity Analysis From Compressed Video Streams

Pixel-level data redundancy in video induces additional memory and computing overhead when neural networks are employed to mine spatiotemporal patterns, e.g. activity and event labels from video streams. This work proposes PixelSieve, to enable highly efficient CNN-based activity analysis directly f...

Full description

Saved in:
Bibliographic Details
Published in:2021 58th ACM/IEEE Design Automation Conference (DAC) pp. 811 - 816
Main Authors: Wang, Yongchen, Wang, Ying, Li, Huawei, Li, Xiaowei
Format: Conference Proceeding
Language:English
Published: IEEE 05.12.2021
Subjects:
Decoding
Design automation
Metadata
Neural networks
Redundancy
Spatiotemporal phenomena
Streaming media
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Pixel-level data redundancy in video induces additional memory and computing overhead when neural networks are employed to mine spatiotemporal patterns, e.g. activity and event labels from video streams. This work proposes PixelSieve, to enable highly efficient CNN-based activity analysis directly from video data in compressed formats. Instead of recovering original RGB frames from compressed video, PixelSieve utilizes the built-in metadata in compressed video streams to distill only the critical pixels that render relevant spatiotemporal features, and then conducts efficient CNN inference with the condensed inputs. PixelSieve removes the overhead of video decoding and significantly improves the performance of CNN-based video analysis by 4.5x on average.
DOI:10.1109/DAC18074.2021.9586310