Effects of Introducing Speech Interaction Modality on Performance of Special Vehicle Crew Under Various Task Complexity Conditions

An experiment with a two interaction modalities (traditional: touch; novel: touch–speech) × two task complexities (low: visual single task; high: audio–visual dual task) within-subjects design was conducted to observe alterations in crew performance (including task performance, subjective workload,...

Full description

Saved in:
Bibliographic Details
Published in:Systems (Basel) Vol. 13; no. 10; p. 847
Main Authors: Feng, Chuanyan, Liu, Shuang, Wanyan, Xiaoru, Qian, Chunying, Ji, Kun, Xie, Fang, Zhou, Yue
Format: Journal Article
Language:English
Published: Basel MDPI AG 01.10.2025
Subjects:
Analysis
Audio data
Chaos theory
Data processing
Displays
Efficiency
Eye movements
eye response
Hypotheses
Information processing
interaction modality
Multitasking
Performance evaluation
Resource allocation
Smartphones
special vehicle
Speech
speech interaction
Speech processing systems
Task complexity
touch interaction
Vehicles
Visual observation
Visual tasks
Workload
Workloads
China
ISSN:2079-8954, 2079-8954
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:An experiment with a two interaction modalities (traditional: touch; novel: touch–speech) × two task complexities (low: visual single task; high: audio–visual dual task) within-subjects design was conducted to observe alterations in crew performance (including task performance, subjective workload, and eye responses) in a typical planning task-based on a high-fidelity special vehicle simulation platform. The results revealed that (1) compared to the traditional interaction modality, the novel interaction modality significantly improved task performance, reduced subjective workload, increased mean peak saccade velocity, and decreased fixation entropy; (2) under high task complexity, subjective workload, mean pupil diameter, and the nearest neighbor index showed significant increases, while no significant decline in task performance was observed; (3) no interaction effect of crew performance was observed between interaction modality and task complexity. The foregoing results imply that (1) the novel interaction modality incorporating speech input exhibits advantages over the traditional touch-based modality in terms of enhancing task performance (over 45% improvement) and reducing cognitive workload; (2) leveraging dual-channel audio–visual information processing facilitates the maintenance of task performance under high task complexity and multi-tasking demands; (3) eye movement characteristics may serve as informative indicators for evaluating the benefits of speech-based interaction and the effectiveness of cognitive resource allocation under high-complexity task conditions. The results can provide a basis for the design of the display and control interface in special vehicles.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2079-8954
2079-8954
DOI:10.3390/systems13100847