Towards Reproducible Language-Based HRI Experiments: Open-Sourcing a Generalized Experiment Project

We are witnessing increasing calls for reproducibility and replicability in HRI studies to improve reliability and confidence in empirical findings. One solution to facilitate this is using a robot platform that researchers frequently use, making it easier to replicate studies to verify results. In...

Full description

Saved in:
Bibliographic Details
Published in:2024 19th ACM/IEEE International Conference on Human-Robot Interaction (HRI) pp. 949 - 953
Main Authors: Tijani, Uthman, Wang, Hong, Han, Zhao
Format: Conference Proceeding
Language:English
Published: ACM 11.03.2024
Subjects:
Choregraphe
Codes
experiment design
Human computer interaction
Human-robot interaction
language-capable robots
Licenses
Open science
Organizations
Reliability
replicability
reproducibility
Reproducibility of results
user studies
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We are witnessing increasing calls for reproducibility and replicability in HRI studies to improve reliability and confidence in empirical findings. One solution to facilitate this is using a robot platform that researchers frequently use, making it easier to replicate studies to verify results. In this work, we focus on a popular, affordable, and rich-in-functionality robot platform, NAO/Pepper, and contribute a generalized experiment project specifically for conducting language-based HRI experiments where a robot instructs a human for a task, including objective data collection.Specifically, we first describe a concrete workflow from an existing experiment and how it is generalized. We then evaluate the generalized project with a case study to show how adopters can quickly adapt to their specific experiment needs. This work provides inspiration for HRI researchers to not only provide their experiment code as supplementary material but also generalize them to benefit other researchers to advance empirical research in HRI. The generalized Choregraphe project with documentation, demo, and usage notes is available under MIT license on GitHub at https://github.com/TheRARELab/langex. We welcome questions by posting GitHub issues and pull requests to share adapted packages.CCS CONCEPTS* Human-centered computing → Systems and tools for interaction design; * Computer systems organization → Robotics.
DOI:10.1145/3610977.3637483