Evaluating the Impact of Personalized Value Alignment in Human-Robot Interaction: Insights into Trust and Team Performance Outcomes

This paper examines the effect of real-time, personalized alignment of a robot's reward function to the human's values on trust and team performance. We present and compare three distinct robot interaction strategies: a non-learner strategy where the robot presumes the human's reward...

Full description

Saved in:
Bibliographic Details
Published in:2024 19th ACM/IEEE International Conference on Human-Robot Interaction (HRI) pp. 32 - 41
Main Authors: Bhat, Shreyas, Lyons, Joseph B., Shi, Cong, Yang, X. Jessie
Format: Conference Proceeding
Language:English
Published: ACM 11.03.2024
Subjects:
Estimation
Human computer interaction
Human-robot interaction
Human-robot teaming
Markov decision processes
Organizations
Real-time systems
Reinforcement learning
trust-aware decision-making
value-alignment
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract This paper examines the effect of real-time, personalized alignment of a robot's reward function to the human's values on trust and team performance. We present and compare three distinct robot interaction strategies: a non-learner strategy where the robot presumes the human's reward function mirrors its own; a non-adaptive-learner strategy in which the robot learns the human's reward function for trust estimation and human behavior modeling, but still optimizes its own reward function; and an adaptive-learner strategy in which the robot learns the human's reward function and adopts it as its own. Two human-subject experiments with a total number of N = 54 participants were conducted. In both experiments, the human-robot team searches for potential threats in a town. The team sequentially goes through search sites to look for threats. We model the interaction between the human and the robot as a trust-aware Markov Decision Process (trust-aware MDP) and use Bayesian Inverse Reinforcement Learning (IRL) to estimate the reward weights of the human as they interact with the robot. In Experiment 1, we start our learning algorithm with an informed prior of the human's values/goals. In Experiment 2, we start the learning algorithm with an uninformed prior. Results indicate that when starting with a good informed prior, personalized value alignment does not seem to benefit trust or team performance. On the other hand, when an informed prior is unavailable, alignment to the human's values leads to high trust and higher perceived performance while maintaining the same objective team performance.CCS CONCEPTS* Human-centered computing → Empirical studies in HCI; * Computer systems organization → Robotic autonomy.
AbstractList This paper examines the effect of real-time, personalized alignment of a robot's reward function to the human's values on trust and team performance. We present and compare three distinct robot interaction strategies: a non-learner strategy where the robot presumes the human's reward function mirrors its own; a non-adaptive-learner strategy in which the robot learns the human's reward function for trust estimation and human behavior modeling, but still optimizes its own reward function; and an adaptive-learner strategy in which the robot learns the human's reward function and adopts it as its own. Two human-subject experiments with a total number of N = 54 participants were conducted. In both experiments, the human-robot team searches for potential threats in a town. The team sequentially goes through search sites to look for threats. We model the interaction between the human and the robot as a trust-aware Markov Decision Process (trust-aware MDP) and use Bayesian Inverse Reinforcement Learning (IRL) to estimate the reward weights of the human as they interact with the robot. In Experiment 1, we start our learning algorithm with an informed prior of the human's values/goals. In Experiment 2, we start the learning algorithm with an uninformed prior. Results indicate that when starting with a good informed prior, personalized value alignment does not seem to benefit trust or team performance. On the other hand, when an informed prior is unavailable, alignment to the human's values leads to high trust and higher perceived performance while maintaining the same objective team performance.CCS CONCEPTS* Human-centered computing → Empirical studies in HCI; * Computer systems organization → Robotic autonomy.
Author Yang, X. Jessie
Bhat, Shreyas
Lyons, Joseph B.
Shi, Cong
Author_xml – sequence: 1
  givenname: Shreyas
  surname: Bhat
  fullname: Bhat, Shreyas
  email: shreyasb@umich.edu
  organization: University of Michigan,Ann Arbor,Michigan,USA
– sequence: 2
  givenname: Joseph B.
  surname: Lyons
  fullname: Lyons, Joseph B.
  email: joseph.lyons.6@us.af.mil
  organization: Air Force Research Laboratory,Dayton,Ohio,USA
– sequence: 3
  givenname: Cong
  surname: Shi
  fullname: Shi, Cong
  email: congshi@bus.miami.edu
  organization: Miami Herbert Business School,Miami,Florida,USA
– sequence: 4
  givenname: X. Jessie
  surname: Yang
  fullname: Yang, X. Jessie
  email: xijyang@umich.edu
  organization: University of Michigan,Ann Arbor,Michigan,USA
BookMark eNotkE1LAzEURSMoqLVrNy7yB0aTybe7UqotFCpS3ZbXmTdtYCYpk4ygW_-4I7q6XLjnLO41OQ8xICG3nN1zLtWD0Jw5Y-6FFtKV_IxMnXFWMmaYKEt1SaYp-T1T0nEmlb0i34sPaAfIPhxoPiJddSeoMo0NfcE-xQCt_8Kavo8jpLPWH0KHIVMf6HLoIBSvcR8zXYWM_cj5GB7HkvzhmNM4ypFu-yFlCqGmW4Tu19rEfiQrpJshV7HDdEMuGmgTTv9zQt6eFtv5slhvnlfz2bqAUtpcgLKGGVuXvJKWOw1l4yQIAw0yDgLAaKOxUhIqY5ysFRjhtGqsrJkRtRQTcvfn9Yi4O_W-g_5zx5keTxNO_ADcEGHL
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1145/3610977.3634921
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Xplore Digital Libary (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9798400703225
EndPage 41
ExternalDocumentID 10661039
Genre orig-research
GrantInformation_xml – fundername: Air Force Office of Scientific Research
  funderid: 10.13039/100000181
GroupedDBID 6IE
6IL
ACM
ALMA_UNASSIGNED_HOLDINGS
APO
CBEJK
LHSKQ
RIE
RIL
ID FETCH-LOGICAL-a248t-a587078d21c48196a2f94a37afe01a3aa7676ec54ac7794d5a73965f84d073d43
IEDL.DBID RIE
ISICitedReferencesCount 9
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001239977500007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Tue May 06 03:31:43 EDT 2025
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a248t-a587078d21c48196a2f94a37afe01a3aa7676ec54ac7794d5a73965f84d073d43
PageCount 10
ParticipantIDs ieee_primary_10661039
PublicationCentury 2000
PublicationDate 2024-March-11
PublicationDateYYYYMMDD 2024-03-11
PublicationDate_xml – month: 03
  year: 2024
  text: 2024-March-11
  day: 11
PublicationDecade 2020
PublicationTitle 2024 19th ACM/IEEE International Conference on Human-Robot Interaction (HRI)
PublicationTitleAbbrev HRI
PublicationYear 2024
Publisher ACM
Publisher_xml – name: ACM
SSID ssib054910458
Score 2.0092578
Snippet This paper examines the effect of real-time, personalized alignment of a robot's reward function to the human's values on trust and team performance. We...
SourceID ieee
SourceType Publisher
StartPage 32
SubjectTerms Estimation
Human computer interaction
Human-robot interaction
Human-robot teaming
Markov decision processes
Organizations
Real-time systems
Reinforcement learning
trust-aware decision-making
value-alignment
Title Evaluating the Impact of Personalized Value Alignment in Human-Robot Interaction: Insights into Trust and Team Performance Outcomes
URI https://ieeexplore.ieee.org/document/10661039
WOSCitedRecordID wos001239977500007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA46PHhSceJvcvDauTRpknoT2VCQOWTKbuMteRkDbWXtPHj1HzfJVsWDB09tQ0ggP_q-916-L4RccKVzdFNMDAObCOV9Vs3AJc5JaYQH9DaLROF7NRjo8TgfrsnqkQuDiPHwGXbCa8zl29IsQ6jM73BvTbo83ySbSqkVWatZPN7PYSHpt5bvYSK75EFKXKkOl0GCj_26PyWaj_7OPzveJe0fIh4dfpuYPbKBxT757K0VuosZ9fCN3kWiIy0dHTbI-gMtffaVkF6_zGcx4U_nBY0R--SxnJY1jaHAFavhyn9UwUmvfKW6pKPAw6BQWDpCeA2tNtwC-rCs_RLFqk2e-r3RzW2yvkshgVToOoFMB10fmzIjPAiQkLpcAFfgsMuAAyipJJpMgFF-i9oMFM9l5rSw_idgBT8graIs8JBQDilqjwMFKBBTDVp2tUNt_DMPedYj0g4jOHlbyWVMmsE7_qP8hGynHimEg12MnZJWvVjiGdky7_W8WpzHSf4CpbSqMw
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV09T8MwELWgIMEEiCK-8cCaUsdO7LAh1KoVpVQooG7VNT5XlSBBbcrAyh_HdlMQAwNTEsuyJX_k3t35PRNyyaVK0IwxyBjoQEjrsyoGJjAmjjNhAb2OPFG4J_t9NRwmg4qs7rkwiOgPn2HDvfpcvi6yhQuV2R1urUmTJ-tkIxIiZEu61mr5WE-HubRfJeDDRHTFnZi4lA0eOxE-9usGFW9A2jv_7HqX1H-oeHTwbWT2yBrm--SzVWl05xNqARzteqojLQwdrLD1B2r6bCshvXmZTnzKn05z6mP2wWMxLkrqg4FLXsO1_Zg7N31uK5UFTR0Tg0KuaYrw6lpdsQvow6K0ixTndfLUbqW3naC6TSGAUKgygEg5ZR8dskxYGBBDaBIBXILBJgMOIGMZYxYJyKTdpDoCyZM4Mkpo-xvQgh-QWl7keEgohxCVRYICJIixAhU3lUGV2WfiMq1HpO5GcPS2FMwYrQbv-I_yC7LVSe97o163f3dCtkOLG9wxL8ZOSa2cLfCMbGbv5XQ-O_cT_gWsNq16
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2024+19th+ACM%2FIEEE+International+Conference+on+Human-Robot+Interaction+%28HRI%29&rft.atitle=Evaluating+the+Impact+of+Personalized+Value+Alignment+in+Human-Robot+Interaction%3A+Insights+into+Trust+and+Team+Performance+Outcomes&rft.au=Bhat%2C+Shreyas&rft.au=Lyons%2C+Joseph+B.&rft.au=Shi%2C+Cong&rft.au=Yang%2C+X.+Jessie&rft.date=2024-03-11&rft.pub=ACM&rft.spage=32&rft.epage=41&rft_id=info:doi/10.1145%2F3610977.3634921&rft.externalDocID=10661039