Offline Evaluation Matters: Investigation of the Influence of Offline Performance on Real-Time Operation of Electromyography-Based Neural-Machine Interfaces

There has been a debate on the most appropriate way to evaluate electromyography (EMG)-based neural-machine interfaces (NMIs). Accordingly, this study examined whether a relationship between offline kinematic predictive accuracy (R2) and user real-time task performance while using the interface coul...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on neural systems and rehabilitation engineering Vol. 31; pp. 680 - 689
Main Authors: Hinson, Robert M., Berman, Joseph, Filer, William, Kamper, Derek, Hu, Xiaogang, Huang, He
Format: Journal Article
Language:English
Published: United States IEEE 2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Artificial neural networks
Completion time
Decoding
Electromyography
Interfaces
Kinematics
machine learning
Matching
Measurement
Motion capture
Myoelectric control
Myoelectricity
Neural networks
neural–machine interface
offline analysis
Performance evaluation
Performance measurement
Posture
Real time operation
real-time performance
Real-time systems
Task analysis
Wrist
ISSN:1534-4320, 1558-0210, 1558-0210
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:There has been a debate on the most appropriate way to evaluate electromyography (EMG)-based neural-machine interfaces (NMIs). Accordingly, this study examined whether a relationship between offline kinematic predictive accuracy (R2) and user real-time task performance while using the interface could be identified. A virtual posture-matching task was developed to evaluate motion capture-based control and myoelectric control with artificial neural networks (ANNs) trained to low (R2 ≈ 0.4), moderate (R2 ≈ 0.6), and high (<inline-formula> <tex-math notation="LaTeX">\text {R}^{\vphantom {\text {D}^{\text {a}}}{2}} \approx 0.8 </tex-math></inline-formula>) offline performance levels. Twelve non-disabled subjects trained with each offline performance level decoder before evaluating final real-time posture matching performance. Moderate to strong relationships were detected between offline performance and all real-time task performance metrics: task completion percentage (r = 0.66, p < 0.001), normalized task completion time (r = −0.51, p = 0.001), path efficiency (r = 0.74, p < 0.001), and target overshoots (r = −0.79, p < 0.001). Significant improvements in each real-time task evaluation metric were also observed between the different offline performance levels. Additionally, subjects rated myoelectric controllers with higher offline performance more favorably. The results of this study support the use and validity of offline analyses for optimization of NMIs in myoelectric control research and development.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:1534-4320
1558-0210
1558-0210
DOI:10.1109/TNSRE.2022.3226229