Automatic Deictic Gestures for Animated Pedagogical Agents

We present a system that automatically generates deictic gestures for animated pedagogical agents (APAs). The system takes audio and text as input, which define what the APA has to say, and generates animated gestures based on a set of rules. The automatically generated gestures point to the exact l...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on learning technologies Vol. 13; no. 1; pp. 1 - 13
Main Authors: Kappagantula, Sri Rama Kartheek, Adamo-Villani, Nicoletta, Wu, Meng-Lin, Popescu, Voicu
Format: Journal Article
Language:English
Published: Piscataway IEEE 01.01.2020
Institute of Electrical and Electronics Engineers, Inc
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Animation
Artificial Intelligence
automatic programming
Automation
CAI
College Faculty
College Students
Computer Assisted Instruction
Distance learning
Educational Psychology
educational simulations
Electronic Learning
gesture
Information Retrieval
instructor interfaces
intelligent agents
learning management systems
Lecture Method
Manuals
Mathematics
Natural Language Processing
Nonverbal Communication
Pedagogy
Scalability
speech analysis
Video Technology
Videos
Visual Aids
ISSN:1939-1382, 2372-0050
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a system that automatically generates deictic gestures for animated pedagogical agents (APAs). The system takes audio and text as input, which define what the APA has to say, and generates animated gestures based on a set of rules. The automatically generated gestures point to the exact locations of elements on a whiteboard nearby the APA, which are calculated by searching for keywords mentioned in the speech. We conducted a study with 100 subjects, in which we compared lecture videos containing gestures automatically-scripted by the system to videos of the same lecture containing manually-scripted gestures. The study results show that the manually-scripted and automatically-scripted lectures had comparable number of gestures, and that the gestures were timed equally well.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1939-1382
2372-0050
DOI:10.1109/TLT.2019.2922134