Multimodality in online education: a comparative study

The commencement of the decade brought along with it a grave pandemic and in response the movement of education forums predominantly into the online world. With a surge in the usage of online video conferencing platforms and tools to better gauge student understanding, there needs to be a mechanism...

Full description

Saved in:
Bibliographic Details
Published in:Multimedia tools and applications Vol. 84; no. 28; pp. 33685 - 33718
Main Authors: Immadisetty, Praneeta, Rajesh, Pooja, Gupta, Akshita, M. R., Dr. Anala, A., Dr. Soumya, Subramanya, Dr. K. N.
Format: Journal Article
Language:English
Published: New York Springer US 01.08.2025
Springer Nature B.V
Subjects:
Classrooms
Comparative studies
Computer Communication Networks
Computer Science
Data Structures and Information Theory
Datasets
Education
Emotions
Eye movements
Feedback
Learning
Machine learning
Multimedia Information Systems
Online instruction
Recognition
Special Purpose and Application-Based Systems
Students
Support vector machines
Teachers
Track 5: Multimedia and Education
Verbal communication
Videoconferencing
Decision-level fusion
Non verbal cues
Custom dataset
Affect recognition
Multimodal architecture
ISSN:1573-7721, 1380-7501, 1573-7721
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The commencement of the decade brought along with it a grave pandemic and in response the movement of education forums predominantly into the online world. With a surge in the usage of online video conferencing platforms and tools to better gauge student understanding, there needs to be a mechanism to assess whether instructors can grasp the extent to which students understand the subject and their response to the educational stimuli. The current systems consider only a single cue with a lack of focus in the educational domain. Thus, there is a necessity for the measurement of an all-encompassing holistic overview of the students’ reaction to the subject matter. This paper highlights the need for a multimodal approach to affect recognition and its deployment in the online classroom while considering four cues, posture and gesture, facial, eye tracking and verbal recognition. In this paper, various machine learning models suitable for each cue are analyzed based on key metrics such as accuracy, ease of data collection, sensitivity, and limitations. The comparison of models revealed the most effective techniques for each cue. A multimodal approach derived from weighted majority voting is proposed by combining the top-performing models for each aspect of student engagement in online learning environments. This paper proposes a novel approach that integrates multiple modalities for affect recognition, aiming to improve the accuracy of student response assessments.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1573-7721
1380-7501
1573-7721
DOI:10.1007/s11042-024-20540-0