Encoder-Decoder Architectures based Video Summarization using Key-Shot Selection Model

With the exponential growth of video data, video summarization has become a challenging task. In this article, we propose a deep learning framework for video summarization that utilizes a sequence learning cum encoder-decoder network architecture with a key-shot selection model. We develop two RNN-b...

Full description

Saved in:
Bibliographic Details
Published in:Multimedia tools and applications Vol. 83; no. 11; pp. 31395 - 31415
Main Authors: Yashwanth, Kolli, Soni, Badal
Format: Journal Article
Language:English
Published: New York Springer US 01.03.2024
Springer Nature B.V
Subjects:
Computer Communication Networks
Computer Science
Data Structures and Information Theory
Datasets
Deep learning
Encoders-Decoders
Multimedia Information Systems
Semantic segmentation
Special Purpose and Application-Based Systems
Video data
Key-Shot
CNN
LSTM
RNN
Deep Learning
Summarization
ISSN:1573-7721, 1380-7501, 1573-7721
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:With the exponential growth of video data, video summarization has become a challenging task. In this article, we propose a deep learning framework for video summarization that utilizes a sequence learning cum encoder-decoder network architecture with a key-shot selection model. We develop two RNN-based deep models, Additive Attentive Summariser (AAS) and Multiplicative Attentive Summariser (MAS), as well as a CNN-based model named - Sequential CNN Summariser (SCS). Our SCS and MAS model displays state-of-the-art performance in semantic segmentation, which we leverage to achieve superior performance in video summarization. We evaluate our models on two well-known datasets, SumMe and TVSum, and show that our proposed MAS and SCS models outperform state-of-the-art models such as DR-DSN. The proposed MAS model achieved an average F1 score of 44.1% and 60.7% on SumMe and TVSum datasets, respectively. Further, our contributions include the development of novel RNN-based and CNN-based models for video summarization and comprehensive experimental evaluations on multiple datasets that demonstrate the effectiveness of our proposed models.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1573-7721
1380-7501
1573-7721
DOI:10.1007/s11042-023-16700-3