Diffusion Probabilistic Modeling for Video Generation

Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation. This paper showcases their ability to sequentially generate video, surpassing prior methods in perceptual and probabilistic forecasting metrics. We propose...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Entropy (Basel, Switzerland) Ročník 25; číslo 10; s. 1469
Hlavní autoři: Yang, Ruihan, Srivastava, Prakhar, Mandt, Stephan
Médium: Journal Article
Jazyk:angličtina
Vydáno: Basel MDPI AG 20.10.2023
MDPI
Témata:
Ablation
Analysis
autoregressive models
Datasets
deep generative models
Diffusion
diffusion models
Forecasting
Image processing
Image quality
Learning models (Stochastic processes)
MATHEMATICS AND COMPUTING
Methods
Physics
Probabilistic models
Probability theory
Variables
Vector space
Video compression
video generation
Video production
United States
ISSN:1099-4300, 1099-4300
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation. This paper showcases their ability to sequentially generate video, surpassing prior methods in perceptual and probabilistic forecasting metrics. We propose an autoregressive, end-to-end optimized video diffusion model inspired by recent advances in neural video compression. The model successively generates future frames by correcting a deterministic next-frame prediction using a stochastic residual generated by an inverse diffusion process. We compare this approach against six baselines on four datasets involving natural and simulation-based videos. We find significant improvements in terms of perceptual quality and probabilistic frame forecasting ability for all datasets.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
SC0022331; 2047418; 2003237; 2007719
USDOE Office of Science (SC)
National Science Foundation (NSF)
ISSN:1099-4300
1099-4300
DOI:10.3390/e25101469