Diffusion Probabilistic Modeling for Video Generation

Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation. This paper showcases their ability to sequentially generate video, surpassing prior methods in perceptual and probabilistic forecasting metrics. We propose...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Entropy (Basel, Switzerland) Ročník 25; číslo 10; s. 1469
Hlavní autori: Yang, Ruihan, Srivastava, Prakhar, Mandt, Stephan
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Basel MDPI AG 20.10.2023
MDPI
Predmet:
Ablation
Analysis
autoregressive models
Datasets
deep generative models
Diffusion
diffusion models
Forecasting
Image processing
Image quality
Learning models (Stochastic processes)
MATHEMATICS AND COMPUTING
Methods
Physics
Probabilistic models
Probability theory
Variables
Vector space
Video compression
video generation
Video production
United States
ISSN:1099-4300, 1099-4300
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation. This paper showcases their ability to sequentially generate video, surpassing prior methods in perceptual and probabilistic forecasting metrics. We propose an autoregressive, end-to-end optimized video diffusion model inspired by recent advances in neural video compression. The model successively generates future frames by correcting a deterministic next-frame prediction using a stochastic residual generated by an inverse diffusion process. We compare this approach against six baselines on four datasets involving natural and simulation-based videos. We find significant improvements in terms of perceptual quality and probabilistic frame forecasting ability for all datasets.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
SC0022331; 2047418; 2003237; 2007719
USDOE Office of Science (SC)
National Science Foundation (NSF)
ISSN:1099-4300
1099-4300
DOI:10.3390/e25101469