Robot Proficiency Self-Assessment Using Assumption-Alignment Tracking

While the design of autonomous robots often emphasizes developing proficient robots, another important attribute of autonomous robot systems is their ability to evaluate their own proficiency. A robot should be able to assess how well it can perform a task before, during, and after it has attempted...

Full description

Saved in:

Bibliographic Details
Published in:	IEEE transactions on robotics Vol. 39; no. 4; pp. 3279 - 3298
Main Authors:	Cao, Xuan, Gautam, Alvika, Whiting, Tim, Smith, Skyler, Goodrich, Michael A., Crandall, Jacob W.
Format:	Journal Article
Language:	English
Published:	New York IEEE 01.08.2023 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:	Algorithms Alignment Assumption-alignment tracking (AAT) autonomous robot system Autonomous robots Decision making Generators Markov chains Performance evaluation robot proficiency self-assessment Robot sensing systems Robots Self assessment Sensors Task analysis Tracking Trajectory
ISSN:	1552-3098, 1941-0468
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	While the design of autonomous robots often emphasizes developing proficient robots, another important attribute of autonomous robot systems is their ability to evaluate their own proficiency. A robot should be able to assess how well it can perform a task before, during, and after it has attempted the task. How can autonomous robots be designed to self-assess their behavior? This article presents the assumption-alignment tracking (AAT) method for designing autonomous robots that can effectively evaluate their own performance. In AAT, the robot a) tracks the veracity of assumptions made by the robot's decision-making algorithms to measure how well these algorithms fit, or align with , its environment and hardware systems, and b) uses the measurement of alignment to assess the robot's ability to succeed at a given task based on its past experiences. The efficacy of AAT is illustrated through three case studies: a simulated robot navigating in a maze-based (discrete time) Markov chain environment, a simulated robot navigating in a continuous environment, and a real-world robot arranging blocks of different shapes and colors in a specific order on a table. Results show that AAT is able to accurately predict robot performance and, hence, determine robot proficiency in real time.
AbstractList	While the design of autonomous robots often emphasizes developing proficient robots, another important attribute of autonomous robot systems is their ability to evaluate their own proficiency. A robot should be able to assess how well it can perform a task before, during, and after it has attempted the task. How can autonomous robots be designed to self-assess their behavior? This article presents the assumption-alignment tracking (AAT) method for designing autonomous robots that can effectively evaluate their own performance. In AAT, the robot a) tracks the veracity of assumptions made by the robot's decision-making algorithms to measure how well these algorithms fit, or align with , its environment and hardware systems, and b) uses the measurement of alignment to assess the robot's ability to succeed at a given task based on its past experiences. The efficacy of AAT is illustrated through three case studies: a simulated robot navigating in a maze-based (discrete time) Markov chain environment, a simulated robot navigating in a continuous environment, and a real-world robot arranging blocks of different shapes and colors in a specific order on a table. Results show that AAT is able to accurately predict robot performance and, hence, determine robot proficiency in real time.
Author	Crandall, Jacob W. Smith, Skyler Gautam, Alvika Cao, Xuan Whiting, Tim Goodrich, Michael A.
Author_xml	– sequence: 1 givenname: Xuan orcidid: 0000-0002-7211-6999 surname: Cao fullname: Cao, Xuan email: caoxuan8872@gmail.com organization: Computer Science Department, Brigham Young University, Provo, UT, USA – sequence: 2 givenname: Alvika surname: Gautam fullname: Gautam, Alvika email: alvikag@tamu.edu organization: Department of Mechanical Engineering, Texas A&M University, College Station, TX, USA – sequence: 3 givenname: Tim orcidid: 0000-0003-4016-1071 surname: Whiting fullname: Whiting, Tim email: tim@whitings.org organization: Computer Science Department, Brigham Young University, Provo, UT, USA – sequence: 4 givenname: Skyler surname: Smith fullname: Smith, Skyler email: skyler.smith897@gmail.com organization: Computer Science Department, Brigham Young University, Provo, UT, USA – sequence: 5 givenname: Michael A. surname: Goodrich fullname: Goodrich, Michael A. email: mike@cs.byu.edu organization: Computer Science Department, Brigham Young University, Provo, UT, USA – sequence: 6 givenname: Jacob W. orcidid: 0000-0002-5602-4146 surname: Crandall fullname: Crandall, Jacob W. email: crandall@cs.byu.edu organization: Computer Science Department, Brigham Young University, Provo, UT, USA
BookMark	eNp9UE1PAjEQbQwmAnr34IHE82I_tu32SAh-JCQYhHPT7bakuLTYLgf-vUU4GA9eZibz3puXeQPQ88EbAO4RHCMExdNquRhjiMmYYIZRxa9AH4kSFbBkVS_PlOKCQFHdgEFKWwhxKSDpg9ky1KEbvcdgnXbG6-Pow7S2mKRkUtoZ343WyfnNKC8Ou33ngi8mrdv4H2gVlf7M6C24tqpN5u7Sh2D9PFtNX4v54uVtOpkXGgvcFdRoa2rWYGyZMkZpapqqEjyXhkBsaUk4Z1aThlNbs0pBoRtWclpzRHmtyBA8nu_uY_g6mNTJbThEny0lrkouGIMEZxY8s3QMKUVj5T66nYpHiaA8hSVzWPIUlryElSXsj0S7Tp2-7aJy7X_Ch7PQGWN--SDIECLkG1rXeXg
CODEN	ITREAE
CitedBy_id	crossref_primary_10_1016_j_cja_2025_103768 crossref_primary_10_1145_3732794 crossref_primary_10_1109_LRA_2023_3316896 crossref_primary_10_1145_3616010 crossref_primary_10_1007_s11431_024_2810_2
Cites_doi	10.1145/3434074.3447181 10.1145/3267338 10.1177/0278364915587924 10.1109/CVPR.2014.456 10.1007/978-3-030-20454-9_21 10.1109/ICRA46639.2022.9812030 10.1109/IV47402.2020.9304844 10.1037/rev0000045 10.15607/RSS.2010.VI.034 10.1016/j.cor.2009.09.001 10.1109/IROS.2016.7759279 10.4108/sas.1.1.e3 10.1109/AERO.2009.4839310 10.2514/6.2016-2310 10.1109/ICRA.2013.6631221 10.1016/j.jss.2017.10.031 10.4028/www.scientific.net/AMR.837.116 10.2514/6.2016-1651 10.1145/3434073.3444657 10.2514/6.2022-2496 10.4028/www.scientific.net/AMR.1036.885 10.1177/1541931215591057 10.1109/LRA.2019.2929999 10.1145/3457185 10.1109/4235.585893 10.1145/3522579 10.3389/fpsyg.2018.00861 10.1108/13552510210439784 10.1109/MIS.2010.126 10.1177/0278364918784361 10.1109/CASE48305.2020.9216965 10.24963/ijcai.2020/669 10.1109/TSMCA.2005.850587 10.1016/j.robot.2022.104136 10.1002/2017EF000660 10.1109/AERO.2004.1367683 10.1109/WI-IAT.2013.157 10.1007/978-3-319-28872-7_26 10.1109/ICASSP.2019.8682730 10.1109/IROS.2011.6094856 10.1109/IJCNN.2013.6706783
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D
DOI	10.1109/TRO.2023.3262187
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Mechanical & Transportation Engineering Abstracts Technology Research Database Engineering Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Mechanical & Transportation Engineering Abstracts Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	1941-0468
EndPage	3298
ExternalDocumentID	10_1109_TRO_2023_3262187 10106113
Genre	orig-research
GrantInformation_xml	– fundername: U.S. Office of Naval Research grantid: #N00014-18-1-2503
GroupedDBID	.DC 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACIWK AENEX AETIX AGQYO AGSQL AHBIQ AIBXA AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ H~9 IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNS VJK AAYXX CITATION 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c292t-5ecfeb6d22f6aeeac5ed8897d88d302f543776fc3d75fb68a09cd6475b7157ba3
IEDL.DBID	RIE
ISICitedReferencesCount	9
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000976067400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1552-3098
IngestDate	Sun Nov 09 05:39:13 EST 2025 Sat Nov 29 01:47:31 EST 2025 Tue Nov 18 22:35:49 EST 2025 Wed Aug 27 02:46:24 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	4
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c292t-5ecfeb6d22f6aeeac5ed8897d88d302f543776fc3d75fb68a09cd6475b7157ba3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-7211-6999 0000-0003-4016-1071 0000-0002-5602-4146
PQID	2847966032
PQPubID	27625
PageCount	20
ParticipantIDs	crossref_primary_10_1109_TRO_2023_3262187 ieee_primary_10106113 proquest_journals_2847966032 crossref_citationtrail_10_1109_TRO_2023_3262187
PublicationCentury	2000
PublicationDate	2023-Aug. 2023-8-00 20230801
PublicationDateYYYYMMDD	2023-08-01
PublicationDate_xml	– month: 08 year: 2023 text: 2023-Aug.
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on robotics
PublicationTitleAbbrev	TRO
PublicationYear	2023
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref12 ref15 ref52 ref10 ref54 fitzgerald (ref44) 0 ref17 ref16 ref19 ref18 zagorecki (ref39) 0 ref51 kempa (ref33) 2014; 837 russell (ref55) 2009 ref46 cao (ref56) 2021 ref45 ref48 ref47 ref42 habbema (ref40) 1976 ref43 béné (ref26) 2018; 6 ref49 ref8 ref7 ref9 ref4 ref3 ref6 (ref57) 2019 ref5 israelsen (ref24) 2019 aitken (ref21) 2016 riemsdijk (ref34) 0; 2 ref35 ref36 kampa (ref29) 2018; 3 ref31 ref32 burghouts (ref13) 0 huang (ref50) 0 ref2 ref1 ref38 morris (ref14) 2007 aitken (ref22) 0 go?da (ref30) 2018; 9 banerjee (ref27) 0 ref23 israelsen (ref20) 0 ref25 ref64 ref63 ref65 frasca (ref11) 0 kaipa (ref41) 0 ref28 wiedemeyer (ref58) 2014 kuter (ref37) 0 bradski (ref59) 0; 25 fletcher (ref53) 2009 sucan (ref60) 2013 ref62 ref61
References_xml	– start-page: 4 year: 0 ident: ref41 article-title: Toward estimating task execution confidence for robotic bin-picking applications publication-title: Proc AAAI Fall Symp Ser – year: 2021 ident: ref56 article-title: Computing agent competency in first order Markov processes – start-page: 399 year: 1976 ident: ref40 article-title: Models for diagnosis and detection of combinations of diseases publication-title: Decision Making and Medical Care – ident: ref7 doi: 10.1145/3434074.3447181 – year: 2019 ident: ref24 article-title: Algorithmic assurances and self-assessment of competency boundaries in autonomous systems – year: 2009 ident: ref55 publication-title: Artificial Intelligence A Modern Approach – ident: ref35 doi: 10.1145/3267338 – ident: ref16 doi: 10.1177/0278364915587924 – ident: ref2 doi: 10.1109/CVPR.2014.456 – ident: ref23 doi: 10.1007/978-3-030-20454-9_21 – ident: ref25 doi: 10.1109/ICRA46639.2022.9812030 – year: 2014 ident: ref58 article-title: IAI Kinect2 – ident: ref19 doi: 10.1109/IV47402.2020.9304844 – start-page: 1 year: 0 ident: ref20 article-title: A factor-based framework for decision-making competency self-assessment publication-title: Proc AAAI SSS-22 Symp Closing Assessment Loop Communicating Proficiency Intent Hum -Robot Teaming – ident: ref45 doi: 10.1037/rev0000045 – start-page: 39 year: 0 ident: ref39 article-title: An approximation of surprise index as a measure of confidence publication-title: Proc AAAI Fall Symp Ser – ident: ref54 doi: 10.15607/RSS.2010.VI.034 – ident: ref31 doi: 10.1016/j.cor.2009.09.001 – ident: ref18 doi: 10.1109/IROS.2016.7759279 – start-page: 1 year: 0 ident: ref11 article-title: Can you do this? Self-assessment dialogues with autonomous robots before, during, and after a mission publication-title: Proc HRI Workshop Assessing Explaining Conveying Robot Proficiency Hum -Robot Teaming – ident: ref49 doi: 10.4108/sas.1.1.e3 – ident: ref8 doi: 10.1109/AERO.2009.4839310 – start-page: 1 year: 0 ident: ref22 article-title: Assurances and machine self-confidence for enhanced trust in autonomous systems publication-title: Proc RSS Workshop Social Trust Auton Syst – ident: ref47 doi: 10.2514/6.2016-2310 – year: 2019 ident: ref57 article-title: Rethink robotics – ident: ref15 doi: 10.1109/ICRA.2013.6631221 – ident: ref65 doi: 10.1016/j.jss.2017.10.031 – volume: 837 start-page: 116 year: 2014 ident: ref33 article-title: Estimation of reliability characteristics in a production scheduling model with failures and time-changing parameters described by gamma and exponential distributions publication-title: Advanced Materials Research doi: 10.4028/www.scientific.net/AMR.837.116 – year: 2013 ident: ref60 article-title: MoveIt motion planning framework – ident: ref38 doi: 10.2514/6.2016-1651 – ident: ref4 doi: 10.1145/3434073.3444657 – volume: 3 start-page: 555 year: 2018 ident: ref29 article-title: The review of reliability factors related to industrial robots publication-title: Robot Autom Eng J – ident: ref42 doi: 10.2514/6.2022-2496 – ident: ref32 doi: 10.4028/www.scientific.net/AMR.1036.885 – ident: ref36 doi: 10.1177/1541931215591057 – ident: ref62 doi: 10.1109/LRA.2019.2929999 – ident: ref5 doi: 10.1145/3457185 – ident: ref52 doi: 10.1109/4235.585893 – volume: 2 start-page: 702 year: 0 ident: ref34 article-title: Goals in agent systems: A unifying framework publication-title: Proc Int Joint Conf Autonomous Agents and Multiagent Systems – ident: ref43 doi: 10.1145/3522579 – start-page: 16 year: 0 ident: ref50 article-title: Autonomy level specification for intelligent autonomous vehicles: Interim progress report publication-title: Proc Perform Metrics Intell Syst Workshop – ident: ref3 doi: 10.3389/fpsyg.2018.00861 – ident: ref28 doi: 10.1108/13552510210439784 – ident: ref46 doi: 10.1109/MIS.2010.126 – ident: ref61 doi: 10.1177/0278364918784361 – start-page: 509 year: 2009 ident: ref53 publication-title: The MIT-Cornell Collision and Why It Happened – ident: ref6 doi: 10.1109/CASE48305.2020.9216965 – ident: ref64 doi: 10.24963/ijcai.2020/669 – ident: ref51 doi: 10.1109/TSMCA.2005.850587 – start-page: 18 year: 0 ident: ref37 article-title: Computational mechanisms to support reporting of self confidence of automated/autonomous systems publication-title: Proc AAAI Fall Symp Ser – ident: ref63 doi: 10.1016/j.robot.2022.104136 – year: 2007 ident: ref14 article-title: Robotic introspection for exploration and mapping of subterranean environments – start-page: 1350 year: 0 ident: ref44 article-title: Human-guided trajectory adaptation for tool transfer publication-title: Proc 18th Int Conf Auton Agents MultiAgent Syst – volume: 6 start-page: 979 year: 2018 ident: ref26 article-title: From resistance to transformation: A generic metric of resilience through viability publication-title: Earth's Future doi: 10.1002/2017EF000660 – volume: 25 start-page: 120 year: 0 ident: ref59 article-title: The OpenCV library publication-title: Dr Dobb's J Softw Tools – ident: ref1 doi: 10.1109/AERO.2004.1367683 – ident: ref12 doi: 10.1109/WI-IAT.2013.157 – start-page: 1 year: 0 ident: ref27 article-title: Fault diagnosis in robot task execution publication-title: Proc AAAI Spring Symp Ser – volume: 9 start-page: 24 year: 2018 ident: ref30 article-title: Analysis of human operators and industrial robots performance and reliability publication-title: Manage Prod Eng Rev – ident: ref17 doi: 10.1007/978-3-319-28872-7_26 – ident: ref48 doi: 10.1109/ICASSP.2019.8682730 – ident: ref9 doi: 10.1109/IROS.2011.6094856 – start-page: 1 year: 0 ident: ref13 article-title: Robotic self-assessment of competence publication-title: Proc HRI Workshop Assessing Explaining Conveying Robot Proficiency Hum -Robot Teaming – ident: ref10 doi: 10.1109/IJCNN.2013.6706783 – year: 2016 ident: ref21 article-title: Assured human-autonomy interaction through machine self-confidence
SSID	ssj0024903
Score	2.4499505
Snippet	While the design of autonomous robots often emphasizes developing proficient robots, another important attribute of autonomous robot systems is their ability...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	3279
SubjectTerms	Algorithms Alignment Assumption-alignment tracking (AAT) autonomous robot system Autonomous robots Decision making Generators Markov chains Performance evaluation robot proficiency self-assessment Robot sensing systems Robots Self assessment Sensors Task analysis Tracking Trajectory
Title	Robot Proficiency Self-Assessment Using Assumption-Alignment Tracking
URI	https://ieeexplore.ieee.org/document/10106113 https://www.proquest.com/docview/2847966032
Volume	39
WOSCitedRecordID	wos000976067400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1941-0468 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0024903 issn: 1552-3098 databaseCode: RIE dateStart: 20040101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA46POjBnxOnU3rw4iFbTZukOQ7Z8DTHnLBbafNeZDA22TrB_94k7eZEFLyUUpJSvtf3Xn68fB8htyAM4zkoKmOe0DgzYH3OBsOYgX3KjKM492ITst9PxmM1qA6r-7MwiOiLz7Dlbv1ePsz1yi2VWQ93ExinUbsrpSgPa30R6ykvg-woxWgUqmS9Jxmq9mj41HIy4S07VrEpTX7LQV5U5Uck9umld_TPDzsmh9U4MuiUhj8hOzg7JQdb7IJnpDuc5_MiGDhVbu_BH8EzTg3tbMg4A18wENgH1qjOQrQznbz6-oDAJjHtltHr5KXXHT080ko1gWqmWEE5aoO5AMaMyNDGVY6QJEraC0QhMzyOLHhGRyC5yUWShUqDiCXP5T2XeRadk9psPsMLEkAmFehE5JmF1wDagAQSZSy0RIgRGqS9xjHVFaW4U7aYpn5qEarUIp865NMK-Qa52_R4K-k0_mhbd0hvtStBbpDm2lZp5XDL1GVZRzQasctful2Rfff2snivSWrFYoXXZE-_F5Pl4sb_S5_QZMdp
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFA4yBfXB68Tp1D744kO2mjZN8zhkY-KcY07YW2hzkcHYZOsE_705aTcnouBLKSWh5Ts95-Ry8n0IXavIEJoqjllIYxwmRlmfs8EwJMo-JQYozp3YBOt24-GQ94rD6u4sjNbaFZ_pGty6vXw1lQtYKrMeDhMY0KjdBOms4rjWF7Ued0LIQCqGA5_Hy11Jn9cH_acaCIXX7GjFJjX2LQs5WZUfsdglmNb-Pz_tAO0VI0mvkZv-EG3oyRHaXeMXPEbN_jSdZl4PdLmdD394z3pscGNFx-m5kgHPPrBmBRvhxnj06ioEPJvGJCykl9FLqzm4a-NCNwFLwkmGqZZGp5EixESJtpGVahXHnNmLCnxiaBgwFhkZKEZNGsWJz6WKQkZTdktZmgQnqDSZTvQp8lTCuJJxlCYWXqO0DUmKaRZGkmkValVB9SWOQhak4qBtMRZucuFzYZEXgLwokK-gm1WPt5xQ44-2ZUB6rV0OcgVVl7YShcvNBeRZoBoNyNkv3a7Qdnvw2BGd--7DOdqBN-WlfFVUymYLfYG25Hs2ms8u3X_1CV58yrI
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Robot+Proficiency+Self-Assessment+Using+Assumption-Alignment+Tracking&rft.jtitle=IEEE+transactions+on+robotics&rft.au=Cao%2C+Xuan&rft.au=Alvika+Gautam&rft.au=Whiting%2C+Tim&rft.au=Smith%2C+Skyler&rft.date=2023-08-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=1552-3098&rft.eissn=1941-0468&rft.volume=39&rft.issue=4&rft.spage=3279&rft_id=info:doi/10.1109%2FTRO.2023.3262187&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1552-3098&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1552-3098&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1552-3098&client=summon