Explainable Human-Machine Teaming using Model Checking and Interpretable Machine Learning

The human-machine teaming paradigm promotes tight teamwork between humans and autonomous machines that collaborate in the same physical space. This paradigm is increasingly widespread in critical domains, such as healthcare and domestic assistance. These systems are expected to build a certain level...

Full description

Saved in:
Bibliographic Details
Published in:FME Workshop on Formal Methods in Software Engineering (Online) pp. 18 - 28
Main Authors: Bersani, Marcello M., Camilli, Matteo, Lestingi, Livia, Mirandola, Raffaela, Rossi, Matteo
Format: Conference Proceeding
Language:English
Published: IEEE 01.05.2023
Subjects:
formal analysis
Human-machine teaming
interpretable machine learning
statistical model checking
ISSN:2575-5099
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract The human-machine teaming paradigm promotes tight teamwork between humans and autonomous machines that collaborate in the same physical space. This paradigm is increasingly widespread in critical domains, such as healthcare and domestic assistance. These systems are expected to build a certain level of trust by enforcing dependability and exhibiting interpretable behavior. However, trustworthiness is negatively affected by the black-box nature of these systems, which typically make fully autonomous decisions that may be confusing for humans or cause hazards in critical domains. We present the EASE approach, whose goal is to build better trust in human-machine teaming leveraging statistical model checking and model-agnostic interpretable machine learning. We illustrate EASE through an example in healthcare featuring an infinite (dense) space of human-machine uncertain factors, such as diverse physical and physiological characteristics of the agents involved in the teamwork. Our evaluation demonstrates the suitability and cost-effectiveness of EASE in explaining dependability properties in human-machine teaming.
AbstractList The human-machine teaming paradigm promotes tight teamwork between humans and autonomous machines that collaborate in the same physical space. This paradigm is increasingly widespread in critical domains, such as healthcare and domestic assistance. These systems are expected to build a certain level of trust by enforcing dependability and exhibiting interpretable behavior. However, trustworthiness is negatively affected by the black-box nature of these systems, which typically make fully autonomous decisions that may be confusing for humans or cause hazards in critical domains. We present the EASE approach, whose goal is to build better trust in human-machine teaming leveraging statistical model checking and model-agnostic interpretable machine learning. We illustrate EASE through an example in healthcare featuring an infinite (dense) space of human-machine uncertain factors, such as diverse physical and physiological characteristics of the agents involved in the teamwork. Our evaluation demonstrates the suitability and cost-effectiveness of EASE in explaining dependability properties in human-machine teaming.
Author Bersani, Marcello M.
Lestingi, Livia
Mirandola, Raffaela
Camilli, Matteo
Rossi, Matteo
Author_xml – sequence: 1
  givenname: Marcello M.
  surname: Bersani
  fullname: Bersani, Marcello M.
  email: Marcello.Bersani@polimi.it
  organization: Politecnico di Milano,Department of Electronics, Information and Bioengineering (DEIB),Italy
– sequence: 2
  givenname: Matteo
  surname: Camilli
  fullname: Camilli, Matteo
  email: Matteo.Camilli@polimi.it
  organization: Politecnico di Milano,Department of Electronics, Information and Bioengineering (DEIB),Italy
– sequence: 3
  givenname: Livia
  surname: Lestingi
  fullname: Lestingi, Livia
  email: Livia.Lestingi@polimi.it
  organization: Politecnico di Milano,Department of Electronics, Information and Bioengineering (DEIB),Italy
– sequence: 4
  givenname: Raffaela
  surname: Mirandola
  fullname: Mirandola, Raffaela
  email: Raffaela.Mirandola@polimi.it
  organization: Politecnico di Milano,Department of Electronics, Information and Bioengineering (DEIB),Italy
– sequence: 5
  givenname: Matteo
  surname: Rossi
  fullname: Rossi, Matteo
  email: matteo.rossi@polimi.it
  organization: Politecnico di Milano,Department of Mechanical Engineering (DMec),Italy
BookMark eNo1jMtOwzAURA0CiVL6ByzCB6Rc27mJvURVSiulYkFZsKoc-5oaErdKUgn-npbHZkYjnTPX7CLuIjF2x2HKOej7-a5rTROeS1S6UFMBQk4BgMMZm-hCK4kgucilOmcjgQWmCFpfsUnfvx8xyTUA4oi9lp_7xoRo6oaSxaE1MV0Zuw2RkjWZNsS35NCfcrVz1CSzLdmP0zTRJcs4ULfvaPiR_7WKTBePyA279KbpafLXY_YyL9ezRVo9PS5nD1UaBOKQChIqLwRJcKiIwHnE2tuceE0g0QieeZFJD2Bt5hTU0lltc6sF5d64TI7Z7e9vIKLNvgut6b42HHieFZLLbxGyWFw
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/FormaliSE58978.2023.00010
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 9798350312638
EISSN 2575-5099
EndPage 28
ExternalDocumentID 10164731
Genre orig-research
GroupedDBID 6IE
6IF
6IL
6IN
AAJGR
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
OCL
RIE
RIL
ID FETCH-LOGICAL-i255t-2e28672e30d58ee0df55bfc6e1be035a214f243f00cc4d80b3dc9c6c92e6fad43
IEDL.DBID RIE
ISICitedReferencesCount 4
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001021640100003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:19:18 EDT 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i255t-2e28672e30d58ee0df55bfc6e1be035a214f243f00cc4d80b3dc9c6c92e6fad43
OpenAccessLink https://hdl.handle.net/11311/1243584
PageCount 11
ParticipantIDs ieee_primary_10164731
PublicationCentury 2000
PublicationDate 2023-May
PublicationDateYYYYMMDD 2023-05-01
PublicationDate_xml – month: 05
  year: 2023
  text: 2023-May
PublicationDecade 2020
PublicationTitle FME Workshop on Formal Methods in Software Engineering (Online)
PublicationTitleAbbrev FORMALISE
PublicationYear 2023
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0003190055
Score 1.8595704
Snippet The human-machine teaming paradigm promotes tight teamwork between humans and autonomous machines that collaborate in the same physical space. This paradigm is...
SourceID ieee
SourceType Publisher
StartPage 18
SubjectTerms formal analysis
Human-machine teaming
interpretable machine learning
statistical model checking
Title Explainable Human-Machine Teaming using Model Checking and Interpretable Machine Learning
URI https://ieeexplore.ieee.org/document/10164731
WOSCitedRecordID wos001021640100003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1La8JAEF5aKaUn-7D0zRZ6Xd1sNtnkLEoPrQi1xZ5kHxMr2ChW-_u7s0btpYdCDiEwEHY27MzkexDyICOTiUgbZpRMmcy5Y77KsEzlViVOKKcCNuftSfV62XCY9yuyeuDCAEAAn0ETb8O_fDezKxyVtaKgfoWs6X2l0jVZaztQ8XsJBaUOyX2lo9nqYtU3nbx0ksz3Sk00Cke5QiTL_nJSCQdJt_7PVzgmjR0lj_a3h80J2YPylNQ3ngy0-kTPyDuC6ipGFA0TevYc8JJAB6A_fSxFqPuYognalLY_wOKwnOrS0R0AEYM3YZUC67hBXrudQfuRVfYJbOL7hCUTILJUCYi5SzIA7ookMYVNITLA40SLSBZCxgXn1kqXcRM7m9vU5gLSQjsZn5NaOSvhglDtc-gLNZfzHKT2FZUwUeYv65s76xugS9LApRrN1woZo80qXf3x_JocYTbWwMEbUlsuVnBLDuz3cvK1uAt5_QEr5qQ5
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT8JAEJ4YNOoJHxjfronXhe12-zoTCEYgJKLBE2l3p0iCYBD8_e4sBbx4MOmhaTJJs7PNzky_B8CD8rJYemnGs0iFXCXCcFtlaB4lOgqMjEzksDmv7ajbjQeDpFeQ1R0XBhEd-AyrdOv-5ZuZXtKorOY59StiTe8GSkmxomttRip2N5Gk1D7cF0qatSbVfZPxcyOIbbdUJatwEiwkuuwvLxV3lDTL_3yJI6hsSXmstzlujmEHpydQXrsysOIjPYU3gtUVnCjmZvS84xCTyPqYfthYRmD3ESMbtAmrv6OmcTlLp4ZtIYgUvA4rNFhHFXhpNvr1Fi8MFPjYdgoLLlHGYSTRFyaIEYXJgyDLdYhehsIPUumpXCo_F0JrZWKR-UYnOtSJxDBPjfLPoDSdTfEcWGqzaEs1k4gEVWprKpl5sb20be-0bYEuoEJLNfxcaWQM16t0-cfzOzho9TvtYfux-3QFh5SZFYzwGkqL-RJvYE9_L8Zf81uX4x_kJKeA
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=FME+Workshop+on+Formal+Methods+in+Software+Engineering+%28Online%29&rft.atitle=Explainable+Human-Machine+Teaming+using+Model+Checking+and+Interpretable+Machine+Learning&rft.au=Bersani%2C+Marcello+M.&rft.au=Camilli%2C+Matteo&rft.au=Lestingi%2C+Livia&rft.au=Mirandola%2C+Raffaela&rft.date=2023-05-01&rft.pub=IEEE&rft.eissn=2575-5099&rft.spage=18&rft.epage=28&rft_id=info:doi/10.1109%2FFormaliSE58978.2023.00010&rft.externalDocID=10164731