Detection of Important Scenes in Baseball Videos via a Time-Lag-Aware Multimodal Variational Autoencoder

A new method for the detection of important scenes in baseball videos via a time-lag-aware multimodal variational autoencoder (Tl-MVAE) is presented in this paper. Tl-MVAE estimates latent features calculated from tweet, video, and audio features extracted from tweets and videos. Then, important sce...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Sensors (Basel, Switzerland) Ročník 21; číslo 6; s. 2045
Hlavní autoři: Hirasawa, Kaito, Maeda, Keisuke, Ogawa, Takahiro, Haseyama, Miki
Médium: Journal Article
Jazyk:angličtina
Vydáno: Switzerland MDPI AG 14.03.2021
MDPI
Témata:
Baseball
detection of important scenes
Influence
Methods
multimodal variational autoencoder
Neural networks
Sensors
Social networks
sports video
time-lags
Twitter
Twitter
time-lags
multimodal variational autoencoder
detection of important scenes
sports video
ISSN:1424-8220, 1424-8220
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:A new method for the detection of important scenes in baseball videos via a time-lag-aware multimodal variational autoencoder (Tl-MVAE) is presented in this paper. Tl-MVAE estimates latent features calculated from tweet, video, and audio features extracted from tweets and videos. Then, important scenes are detected by estimating the probability of the scene being important from estimated latent features. It should be noted that there exist time-lags between tweets posted by users and videos. To consider the time-lags between tweet features and other features calculated from corresponding multiple previous events, the feature transformation based on feature correlation considering such time-lags is newly introduced to the encoder in MVAE in the proposed method. This is the biggest contribution of the Tl-MVAE. Experimental results obtained from actual baseball videos and their corresponding tweets show the effectiveness of the proposed method.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
This paper is an extended version of our paper published in: Hirasawa, K.; Maeda, K.; Ogawa, T.; Haseyama, M. Important Scene Detection Based on Anomaly Detection using Long Short-Term Memory for Baseball Highlight Generation. In the Proceedings of the IEEE International Conference on Consumer Electronics—Taiwan (IEEE 2020 ICCE-TW), Taoyuan, Taiwan, 28–30 September 2020.
ISSN:1424-8220
1424-8220
DOI:10.3390/s21062045