Automatic Generation of Multimedia Teaching Materials Based on Generative AI: Taking Tang Poetry as an Example

Generative artificial intelligence (AI) is widely recognized as one of the most influential technologies for the future, having sparked a paradigm shift in scientific research. The field of education has also been greatly impacted by this transformative technology, with researchers exploring the app...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on learning technologies Vol. 17; pp. 1327 - 1340
Main Authors: Chen, Xu, Wu, Di
Format: Journal Article
Language:English
Published: Piscataway IEEE 2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Artificial intelligence
Automatic generation
Comparative Analysis
Computer Assisted Instruction
Education
Educational Resources
Empirical analysis
Generative AI
Generative artificial intelligence
generative artificial intelligence (AI)
Image processing
Instructional Materials
Interpolation
Learning Processes
Modules
Multimedia
Multimedia Materials
multimedia teaching materials
Poetry
Scientific Research
Semantics
Solid modeling
Speech recognition
Tang poetry situational videos
Teaching Methods
Video
Video Technology
Videos
Visualization
ISSN:1939-1382, 2372-0050
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Generative artificial intelligence (AI) is widely recognized as one of the most influential technologies for the future, having sparked a paradigm shift in scientific research. The field of education has also been greatly impacted by this transformative technology, with researchers exploring the applications of generative AI, particularly ChatGPT, in education. However, existing research primarily focuses on generating text from text, and there remains a relative scarcity of studies on leveraging multimodal generation capabilities to address key challenges in multimodal data supported instruction. In this article, we present a technical framework for generating Tang poetry situational videos, emphasizing the utilization of generative AI to address the need for multimedia teaching resources. Our framework comprises three main modules: textual situational comprehension, image creation, and video generation. Moreover, we have developed a situational video generation system that incorporates various technologies, including text-to-text generation models, text-to-image generation models, image interpolation, text-to-speech synthesis, and video synthesis. To ascertain the efficacy of the modules within the Tang poetry situational video generation system, we undertook a comparative analysis utilizing the prevalent text-to-image and text-to-video generation models. The empirical findings indicate that our approach is capable of generating images that exhibit greater semantic similarity with the poems, thereby enabling a better comprehension of the poem's connotations and its key components. Concurrently, the Tang poetry videos generated can significantly contribute to the reduction of cognitive load and the enhancement of understanding during the learning process. Our research showcases the potential of generative AI in the education field, specifically in the domain of multimodal teaching resources.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1939-1382
2372-0050
DOI:10.1109/TLT.2024.3378279