Diffusion Probabilistic Modeling for Video Generation

Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation. This paper showcases their ability to sequentially generate video, surpassing prior methods in perceptual and probabilistic forecasting metrics. We propose...

Full description

Saved in:
Bibliographic Details
Published in:Entropy (Basel, Switzerland) Vol. 25; no. 10; p. 1469
Main Authors: Yang, Ruihan, Srivastava, Prakhar, Mandt, Stephan
Format: Journal Article
Language:English
Published: Basel MDPI AG 20.10.2023
MDPI
Subjects:
Ablation
Analysis
autoregressive models
Datasets
deep generative models
Diffusion
diffusion models
Forecasting
Image processing
Image quality
Learning models (Stochastic processes)
MATHEMATICS AND COMPUTING
Methods
Physics
Probabilistic models
Probability theory
Variables
Vector space
Video compression
video generation
Video production
United States
ISSN:1099-4300, 1099-4300
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation. This paper showcases their ability to sequentially generate video, surpassing prior methods in perceptual and probabilistic forecasting metrics. We propose an autoregressive, end-to-end optimized video diffusion model inspired by recent advances in neural video compression. The model successively generates future frames by correcting a deterministic next-frame prediction using a stochastic residual generated by an inverse diffusion process. We compare this approach against six baselines on four datasets involving natural and simulation-based videos. We find significant improvements in terms of perceptual quality and probabilistic frame forecasting ability for all datasets.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
SC0022331; 2047418; 2003237; 2007719
USDOE Office of Science (SC)
National Science Foundation (NSF)
ISSN:1099-4300
1099-4300
DOI:10.3390/e25101469