Video description: A comprehensive survey of deep learning approaches

Video description refers to understanding visual content and transforming that acquired understanding into automatic textual narration. It bridges the key AI fields of computer vision and natural language processing in conjunction with real-time and practical applications. Deep learning-based approa...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:The Artificial intelligence review Jg. 56; H. 11; S. 13293 - 13372
Hauptverfasser: Rafiq, Ghazala, Rafiq, Muhammad, Choi, Gyu Sang
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Dordrecht Springer Netherlands 01.11.2023
Springer
Springer Nature B.V
Schlagworte:
Artificial Intelligence
Attention
Coders
Computational linguistics
Computer Science
Computer vision
Deep learning
Dependency
Distinctiveness
Exploitation
Image processing
Language processing
Learning
Machine learning
Machine vision
Methods
Narration
Natural language interfaces
Natural language processing
Recurrence
Reinforcement
Subtitles & subtitling
Summarization
Surveillance
Surveys
Transformers
Variants
Video data
Deep learning
Text description
Encoder–Decoder architecture
Video description approaches
Video captioning
Video captioning techniques
Vision to text
ISSN:0269-2821, 1573-7462
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Video description refers to understanding visual content and transforming that acquired understanding into automatic textual narration. It bridges the key AI fields of computer vision and natural language processing in conjunction with real-time and practical applications. Deep learning-based approaches employed for video description have demonstrated enhanced results compared to conventional approaches. The current literature lacks a thorough interpretation of the recently developed and employed sequence to sequence techniques for video description. This paper fills that gap by focusing mainly on deep learning-enabled approaches to automatic caption generation. Sequence to sequence models follow an Encoder–Decoder architecture employing a specific composition of CNN, RNN, or the variants LSTM or GRU as an encoder and decoder block. This standard-architecture can be fused with an attention mechanism to focus on a specific distinctiveness, achieving high quality results. Reinforcement learning employed within the Encoder–Decoder structure can progressively deliver state-of-the-art captions by following exploration and exploitation strategies. The transformer mechanism is a modern and efficient transductive architecture for robust output. Free from recurrence, and solely based on self-attention, it allows parallelization along with training on a massive amount of data. It can fully utilize the available GPUs for most NLP tasks. Recently, with the emergence of several versions of transformers, long term dependency handling is not an issue anymore for researchers engaged in video processing for summarization and description, or for autonomous-vehicle, surveillance, and instructional purposes. They can get auspicious directions from this research.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0269-2821
1573-7462
DOI:10.1007/s10462-023-10414-6