SACSoN: Scalable Autonomous Control for Social Navigation

Machine learning provides a powerful tool for building socially compliant robotic systems that go beyond simple predictive models of human behavior. By observing and understanding human interactions from past experiences, learning can enable effective social navigation behaviors directly from data....

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE robotics and automation letters Ročník 9; číslo 1; s. 49 - 56
Hlavní autori:	Hirose, Noriaki, Shah, Dhruv, Sridhar, Ajay, Levine, Sergey
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	Piscataway IEEE 01.01.2024 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:	Behavioral sciences Data Sets for Robot Learning Datasets Human behavior Machine learning Machine Learning for Robot Control Navigation Pedestrians Perturbation Perturbation methods Policies Prediction models Predictive models Robot control Robot learning Robots social navigation Social robots Visual observation Visualization
ISSN:	2377-3766, 2377-3766
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	Machine learning provides a powerful tool for building socially compliant robotic systems that go beyond simple predictive models of human behavior. By observing and understanding human interactions from past experiences, learning can enable effective social navigation behaviors directly from data. In this letter, our goal is to develop methods for training policies for socially unobtrusive behavior, such that robots can navigate among humans in ways that don't disturb human behavior in visual navigation using only onboard RGB observations. We introduce a definition for such behavior based on the counterfactual perturbation of the human: If the robot had not intruded into the space, would the human have acted in the same way? By minimizing this counterfactual perturbation, we can induce robots to behave in ways that do not alter the natural behavior of humans in the shared space. Instantiating this principle requires training policies to minimize their effect on human behavior, and this in turn requires data that allows us to model the behavior of humans in the presence of robots. Therefore, our approach is based on two key contributions. First, we collect a large dataset where an indoor mobile robot interacts with human bystanders. Second, we utilize this dataset to train policies that minimize counterfactual perturbation. We provide supplementary videos and make publicly available the visual navigation dataset on our project page.
AbstractList	Machine learning provides a powerful tool for building socially compliant robotic systems that go beyond simple predictive models of human behavior. By observing and understanding human interactions from past experiences, learning can enable effective social navigation behaviors directly from data. In this letter, our goal is to develop methods for training policies for socially unobtrusive behavior, such that robots can navigate among humans in ways that don't disturb human behavior in visual navigation using only onboard RGB observations. We introduce a definition for such behavior based on the counterfactual perturbation of the human: If the robot had not intruded into the space, would the human have acted in the same way? By minimizing this counterfactual perturbation, we can induce robots to behave in ways that do not alter the natural behavior of humans in the shared space. Instantiating this principle requires training policies to minimize their effect on human behavior, and this in turn requires data that allows us to model the behavior of humans in the presence of robots. Therefore, our approach is based on two key contributions. First, we collect a large dataset where an indoor mobile robot interacts with human bystanders. Second, we utilize this dataset to train policies that minimize counterfactual perturbation. We provide supplementary videos and make publicly available the visual navigation dataset on our project page.
Author	Hirose, Noriaki Levine, Sergey Sridhar, Ajay Shah, Dhruv
Author_xml	– sequence: 1 givenname: Noriaki orcidid: 0000-0003-0361-7383 surname: Hirose fullname: Hirose, Noriaki email: noriaki.hirose@berkeley.edu organization: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA, USA – sequence: 2 givenname: Dhruv orcidid: 0000-0002-7541-3278 surname: Shah fullname: Shah, Dhruv email: shah@eecs.berkeley.edu organization: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA, USA – sequence: 3 givenname: Ajay orcidid: 0009-0008-2268-6097 surname: Sridhar fullname: Sridhar, Ajay email: ajaysridhar@berkeley.edu organization: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA, USA – sequence: 4 givenname: Sergey orcidid: 0000-0001-6764-2743 surname: Levine fullname: Levine, Sergey email: svlevine@eecs.berkeley.edu organization: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA, USA
BookMark	eNp9kMFLwzAUh4NMcM7dPXgoeO5M8tq08VaKU2FMsHoOaZpKRtfMJBX87-3cDsODp_fg_b73Ht8lmvS21whdE7wgBPO71WuxoJjCAoByRtkZmlLIshgyxiYn_QWae7_BGJOUZsDTKeJVUVZ2fR9VSnay7nRUDMH2dmsHH5W2D852UWtdVFllZBet5Zf5kMHY_gqdt7Lzen6sM_S-fHgrn-LVy-NzWaxiRXEWYpIprHhCSdIQSigkjNYtNCkHkrdpDYTocZbUSQNJLllD67SlqZIc6iRXsoEZuj3s3Tn7OWgfxMYOrh9PCppzzDkA5GOKHVLKWe-dboUy4ffP4KTpBMFib0qMpsTelDiaGkH8B9w5s5Xu-z_k5oAYrfVJHPBoFcMP-2xy6A
CODEN	IRALC6
CitedBy_id	crossref_primary_10_1109_ACCESS_2025_3548134 crossref_primary_10_1109_LRA_2025_3595038 crossref_primary_10_3390_s24092794 crossref_primary_10_1109_LRA_2024_3511409 crossref_primary_10_1016_j_commtr_2025_100207 crossref_primary_10_1109_LRA_2025_3572815
Cites_doi	10.1109/TPAMI.2021.3070543 10.1109/ICRA48891.2023.10161227 10.1109/IROS.2016.7759200 10.1007/s10514-023-10103-x 10.1103/PhysRevE.51.4282 10.1145/3583741 10.15607/RSS.2022.XVIII.019 10.1109/IROS.2012.6385716 10.1109/ICRA48891.2023.10160715 10.1007/s11370-020-00324-9 10.1109/ICIP.2017.8296962 10.1109/LRA.2022.3184025 10.1177/0278364915614638 10.1109/IROS.2018.8593871 10.1109/IROS51168.2021.9636319 10.1109/ARSO54254.2022.9802981 10.1109/TCDS.2017.2751963 10.1145/1957656.1957786 10.1109/AIM.2017.8014190 10.1109/LRA.2019.2925731 10.1177/0278364913503892 10.1117/12.823843 10.1109/ICRA48891.2023.10161504 10.1109/CVPR.2016.91 10.1109/IROS.2017.8202312 10.1109/ICRA.2017.7989037 10.1109/IROS47612.2022.9981958 10.1109/CVPR.2012.6248074 10.1109/ICRA.2019.8794457 10.1109/TRO.2007.904911 10.1109/ICRA40945.2020.9196644 10.1109/LRA.2020.2965416 10.1109/IROS.2013.6696576 10.1109/IROS.2009.5354147 10.1109/ICRA.2019.8794134 10.1109/IROS55552.2023.10342447 10.1109/IROS.2016.7759329 10.1109/IROS55552.2023.10341954 10.1109/ICRA48891.2023.10160761 10.3390/app8112205
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/LRA.2023.3329626
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Xplore url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	2377-3766
EndPage	56
ExternalDocumentID	10_1109_LRA_2023_3329626 10305270
Genre	orig-research
GrantInformation_xml	– fundername: ARL DCIST CRA grantid: W911NF-17-2-0181 – fundername: Berkeley DeepDrive at the University of California – fundername: Berkeley and Toyota Motor North America, Inc.
GroupedDBID	0R~ 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF KQ8 M43 M~E O9- OCL RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c207t-17c0c94214d12123462bf3d59318f5b311e2144b4d348a6d2b5f25ca93b48cad3
IEDL.DBID	RIE
ISSN	2377-3766
IngestDate	Sun Nov 30 05:30:37 EST 2025 Sat Nov 29 06:03:27 EST 2025 Tue Nov 18 22:18:56 EST 2025 Wed Aug 27 02:29:06 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	1
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c207t-17c0c94214d12123462bf3d59318f5b311e2144b4d348a6d2b5f25ca93b48cad3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0001-6764-2743 0009-0008-2268-6097 0000-0002-7541-3278 0000-0003-0361-7383
PQID	2890993338
PQPubID	4437225
PageCount	8
ParticipantIDs	crossref_citationtrail_10_1109_LRA_2023_3329626 proquest_journals_2890993338 crossref_primary_10_1109_LRA_2023_3329626 ieee_primary_10305270
PublicationCentury	2000
PublicationDate	2024-Jan. 2024-1-00 20240101
PublicationDateYYYYMMDD	2024-01-01
PublicationDate_xml	– month: 01 year: 2024 text: 2024-Jan.
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE robotics and automation letters
PublicationTitleAbbrev	LRA
PublicationYear	2024
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref12 ref15 ref14 ref52 ref11 Chebotar (ref46) 2021; 139 ref17 ref16 ref19 ref18 Xiao (ref21) 2023 ref51 Kim (ref38) 2022 Mavrogiannis (ref20) 2018 ref45 ref48 ref47 ref42 ref41 ref44 ref43 ref8 ref7 ref9 ref4 ref3 ref6 ref5 ref40 Anderson (ref49) 2018 ref34 Shah (ref10) 2021 ref37 ref36 ref31 ref30 ref33 ref32 ref2 ref1 ref39 Francis (ref50) 2023 Savinov (ref35) 2018 ref24 ref23 ref26 ref25 ref22 ref28 ref27 ref29
References_xml	– ident: ref3 doi: 10.1109/TPAMI.2021.3070543 – ident: ref39 doi: 10.1109/ICRA48891.2023.10161227 – start-page: 393 volume-title: Proc. 6th Conf. Robot Learn. year: 2022 ident: ref38 article-title: Topological Semantic Graph Memory for Image Goal Navigation – start-page: 674 volume-title: Proc. 5th Annu. Conf. Robot Learn. year: 2021 ident: ref10 article-title: Rapid exploration for open-world navigation with latent goal models – ident: ref15 doi: 10.1109/IROS.2016.7759200 – ident: ref23 doi: 10.1007/s10514-023-10103-x – ident: ref1 doi: 10.1103/PhysRevE.51.4282 – ident: ref12 doi: 10.1145/3583741 – ident: ref37 doi: 10.15607/RSS.2022.XVIII.019 – ident: ref16 doi: 10.1109/IROS.2012.6385716 – ident: ref32 doi: 10.1109/ICRA48891.2023.10160715 – ident: ref9 doi: 10.1007/s11370-020-00324-9 – start-page: 1708 volume-title: Proc. Conf. Robot Learn. year: 2023 ident: ref21 article-title: Learning model predictive controllers with real-time attention for real-world navigation – ident: ref43 article-title: Pedestrian detection and tracking by yolov5 and deepsort – ident: ref42 doi: 10.1109/ICIP.2017.8296962 – ident: ref4 doi: 10.1109/LRA.2022.3184025 – ident: ref7 doi: 10.1177/0278364915614638 – year: 2018 ident: ref49 article-title: On evaluation of embodied navigation agents – ident: ref30 doi: 10.1109/IROS.2018.8593871 – ident: ref48 doi: 10.1109/IROS51168.2021.9636319 – start-page: 361 volume-title: Proc. ACM/IEEE Int. Conf. Hum.-Robot Interact. year: 2018 ident: ref20 article-title: Social momentum: A framework for legible navigation in dynamic multi-agent environments – ident: ref51 doi: 10.1109/ARSO54254.2022.9802981 – ident: ref22 doi: 10.1109/TCDS.2017.2751963 – ident: ref14 doi: 10.1145/1957656.1957786 – ident: ref26 doi: 10.1109/AIM.2017.8014190 – ident: ref8 doi: 10.1109/LRA.2019.2925731 – ident: ref33 doi: 10.1177/0278364913503892 – volume-title: Proc. Int. Conf. Learn. Representations year: 2018 ident: ref35 article-title: Semi-parametric topological memory for navigation – ident: ref52 doi: 10.1117/12.823843 – ident: ref47 article-title: Roomba Drivers – ident: ref25 doi: 10.1109/ICRA48891.2023.10161504 – ident: ref40 doi: 10.1109/CVPR.2016.91 – ident: ref28 doi: 10.1109/IROS.2017.8202312 – ident: ref29 doi: 10.1109/ICRA.2017.7989037 – ident: ref44 doi: 10.1109/IROS47612.2022.9981958 – ident: ref6 doi: 10.1109/CVPR.2012.6248074 – ident: ref18 doi: 10.1109/ICRA.2019.8794457 – ident: ref13 doi: 10.1109/TRO.2007.904911 – ident: ref36 doi: 10.1109/ICRA40945.2020.9196644 – ident: ref45 article-title: Ros wrapper for alvar, an open source AR tag tracking library – ident: ref5 doi: 10.1109/LRA.2020.2965416 – year: 2023 ident: ref50 article-title: Principles and guidelines for evaluating social robot navigation algorithms – ident: ref2 doi: 10.1109/IROS.2013.6696576 – ident: ref17 doi: 10.1109/IROS.2009.5354147 – volume: 139 start-page: 1518 volume-title: Proc. 38th Int. Conf. Mach. Learn. year: 2021 ident: ref46 article-title: Actionable models: Unsupervised offline reinforcement learning of robotic skills – ident: ref31 doi: 10.1109/ICRA.2019.8794134 – ident: ref11 doi: 10.1109/IROS55552.2023.10342447 – ident: ref19 doi: 10.1109/IROS.2016.7759329 – ident: ref24 doi: 10.1109/IROS55552.2023.10341954 – ident: ref34 doi: 10.1109/ICRA48891.2023.10160761 – ident: ref27 doi: 10.3390/app8112205 – ident: ref41 article-title: Object detection by yolov5
SSID	ssj0001527395
Score	2.3391533
Snippet	Machine learning provides a powerful tool for building socially compliant robotic systems that go beyond simple predictive models of human behavior. By...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	49
SubjectTerms	Behavioral sciences Data Sets for Robot Learning Datasets Human behavior Machine learning Machine Learning for Robot Control Navigation Pedestrians Perturbation Perturbation methods Policies Prediction models Predictive models Robot control Robot learning Robots social navigation Social robots Visual observation Visualization
Title	SACSoN: Scalable Autonomous Control for Social Navigation
URI	https://ieeexplore.ieee.org/document/10305270 https://www.proquest.com/docview/2890993338
Volume	9
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Xplore customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: RIE dateStart: 20160101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: M~E dateStart: 20160101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8MwGA5ueNCDnxOnc-TgxUO3NEmbxVsZGx60iFPYraRJCoK0snU7-tvNR6cDUfDWQ1LKk6TvV57nBeAaq1CiCKHABLQmQGFUBHksdIAE1bhgIowL32yCpeloPuePDVndcWG01u7ymR7YR1fLV5Vc2VTZ0LbEijAzEXqLMebJWt8JFSslxqNNKRLx4f1TMrDdwQeEYB5b-YQt0-N6qfz4ATurMj385_ccgYPGfYSJX-9jsKPLE7C_JSp4CvgsGc-q9BbODPyWGAWTVW2pCybGh2N_Mx0aVxV6Zi5MxdrJbFRlB7xMJ8_ju6BpkBBIjFgdhEwiySkOqQqtCaIxzguiIm4OahHlJAy1VUTLqSJ0JGKF86jAkRSc5HQkhSJnoF1WpT4HUAqsFXdE2pwWseRWpYdwVHAkjFcUd8Fwg10mG_Vw28TiLXNRBOKZQTuzaGcN2l1w8zXj3Stn_DG2Y9HdGueB7YLeZn2y5mwtM1saNV6Via0vfpl2CfbM26nPlPRAu16s9BXYlev6dbnog9bDx6TvNs8ndzW-kQ
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFA8yBfXgtzidmoMXD93SfLSLtzIcE2cRN2G3kKYpCLLK1u3vNx-dDkTBWw8JLb-X9L2Xl9_vAXCD81AhhlBgElqToMRUBlkkdYAk1biIZRgVvtlEnKbdyYQ_12R1x4XRWrvLZ7ptH10tPy_Vwh6VdWxLLIZjk6FvMkpx6Ola30cqVkyMs1UxEvHO8CVp2_7gbUIwj6yAwprzcd1UfvyCnV_p7__ziw7AXh1AwsRb_BBs6OkR2F2TFTwGfJT0RmV6B0fGAJYaBZNFZckLJsuHPX83HZpgFXpuLkzl0gltlNMT8Nq_H_cGQd0iIVAYxVUQxgopbqCgeWidEI1wVpCccbNVC5aRMNRWEy2jOaFdGeU4YwVmSnKS0a6SOTkFjWk51WcAKol1zh2VNqNFpLjV6SEcFRxJExdFTdBZYSdUrR9u21i8C5dHIC4M2sKiLWq0m-D2a8aH1874Y-yJRXdtnAe2CVor-4h6d82FLY6auMpk1-e_TLsG24Px01AMH9LHC7Bj3kT9uUkLNKrZQl-CLbWs3uazK7eEPgEL2sCn
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=SACSoN%3A+Scalable+Autonomous+Control+for+Social+Navigation&rft.jtitle=IEEE+robotics+and+automation+letters&rft.au=Hirose%2C+Noriaki&rft.au=Shah%2C+Dhruv&rft.au=Sridhar%2C+Ajay&rft.au=Levine%2C+Sergey&rft.date=2024-01-01&rft.pub=IEEE&rft.eissn=2377-3766&rft.volume=9&rft.issue=1&rft.spage=49&rft.epage=56&rft_id=info:doi/10.1109%2FLRA.2023.3329626&rft.externalDocID=10305270
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2377-3766&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2377-3766&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2377-3766&client=summon