CNN in MRF: Video Object Segmentation via Inference in a CNN-Based Higher-Order Spatio-Temporal MRF

This paper addresses the problem of video object segmentation, where the initial object mask is given in the first frame of an input video. We propose a novel spatiotemporal Markov Random Field (MRF) model defined over pixels to handle this problem. Unlike conventional MRF models, the spatial depend...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition s. 5977 - 5986
Hlavní autoři: Bao, Linchao, Wu, Baoyuan, Liu, Wei
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.06.2018
Témata:
Approximation algorithms
Benchmark testing
Inference algorithms
Labeling
Object segmentation
Random variables
Task analysis
ISSN:1063-6919
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This paper addresses the problem of video object segmentation, where the initial object mask is given in the first frame of an input video. We propose a novel spatiotemporal Markov Random Field (MRF) model defined over pixels to handle this problem. Unlike conventional MRF models, the spatial dependencies among pixels in our model are encoded by a Convolutional Neural Network (CNN). Specifically, for a given object, the probability of a labeling to a set of spatially neighboring pixels can be predicted by a CNN trained for this specific object. As a result, higher-order, richer dependencies among pixels in the set can be implicitly modeled by the CNN. With temporal dependencies established by optical flow, the resulting MRF model combines both spatial and temporal cues for tackling video object segmentation. However, performing inference in the MRF model is very difficult due to the very high-order dependencies. To this end, we propose a novel CNN-embedded algorithm to perform approximate inference in the MRF. This algorithm proceeds by alternating between a temporal fusion step and a feed-forward CNN step. When initialized with an appearance-based one-shot segmentation CNN, our model outperforms the winning entries of the DAVIS 2017 Challenge, without resorting to model ensembling or any dedicated detectors.
ISSN:1063-6919
DOI:10.1109/CVPR.2018.00626