In EDS ansehen

PADTHAI‐MM: Principles‐based approach for designing trustworthy, human‐centered AI using the MAST methodology.

Gespeichert in:
Bibliographische Detailangaben
Titel: PADTHAI‐MM: Principles‐based approach for designing trustworthy, human‐centered AI using the MAST methodology.
Autoren: Cohen, Myke C., Kim, Nayoung, Ba, Yang, Pan, Anna, Bhatti, Shawaiz, Salehi, Pouria, Sung, James, Blasch, Erik, Mancenido, Mickey V., Chiou, Erin K.
Quelle: AI Magazine; Mar2025, Vol. 46 Issue 1, p1-34, 34p
Schlagwörter: DECISION support systems, ARTIFICIAL intelligence, SYSTEMS design, INTELLIGENCE service, TRUST, NATURAL language processing
Abstract: Despite an extensive body of literature on trust in technology, designing trustworthy AI systems for high‐stakes decision domains remains a significant challenge. Widely used system design guidelines and tools are rarely attuned to domain‐specific trustworthiness principles. In this study, we introduce a design framework to address this gap within intelligence analytic tasks, called the Principles‐based Approach for Designing Trustworthy, Human‐centered AI using the MAST Methodology (PADTHAI‐MM). PADTHAI‐MM builds on the Multisource AI Scorecard Table (MAST), an AI decision support system evaluation tool designed in accordance to the U.S. Intelligence Community's standards for system trustworthiness. We demonstrate PADTHAI‐MM in our development of the Reporting Assistant for Defense and Intelligence Tasks (READIT), a research platform that leverages data visualizations and natural language processing‐based text analysis to emulate AI‐enabled intelligence reporting aids. To empirically assess the efficacy of PADTHAI‐MM, we developed two versions of READIT for comparison: a "High‐MAST" version, which incorporates AI contextual information and explanations, and a "Low‐MAST" version, designed to be akin to inscrutable "black box" AI systems. Through an iterative design process guided by stakeholder feedback, our multidisciplinary design team developed prototypes that were evaluated by experienced intelligence analysts. Results substantially supported the viability of PADTHAI‐MM in designing for system trustworthiness in this task domain. We also explored the relationship between analysts' MAST ratings and three theoretical categories of information known to impact trust: process, purpose, and performance. Overall, our study supports the practical and theoretical viability of PADTHAI‐MM as an approach to designing trustable AI systems. [ABSTRACT FROM AUTHOR]
Copyright of AI Magazine is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Datenbank: Complementary Index
Beschreibung
Abstract:Despite an extensive body of literature on trust in technology, designing trustworthy AI systems for high‐stakes decision domains remains a significant challenge. Widely used system design guidelines and tools are rarely attuned to domain‐specific trustworthiness principles. In this study, we introduce a design framework to address this gap within intelligence analytic tasks, called the Principles‐based Approach for Designing Trustworthy, Human‐centered AI using the MAST Methodology (PADTHAI‐MM). PADTHAI‐MM builds on the Multisource AI Scorecard Table (MAST), an AI decision support system evaluation tool designed in accordance to the U.S. Intelligence Community's standards for system trustworthiness. We demonstrate PADTHAI‐MM in our development of the Reporting Assistant for Defense and Intelligence Tasks (READIT), a research platform that leverages data visualizations and natural language processing‐based text analysis to emulate AI‐enabled intelligence reporting aids. To empirically assess the efficacy of PADTHAI‐MM, we developed two versions of READIT for comparison: a "High‐MAST" version, which incorporates AI contextual information and explanations, and a "Low‐MAST" version, designed to be akin to inscrutable "black box" AI systems. Through an iterative design process guided by stakeholder feedback, our multidisciplinary design team developed prototypes that were evaluated by experienced intelligence analysts. Results substantially supported the viability of PADTHAI‐MM in designing for system trustworthiness in this task domain. We also explored the relationship between analysts' MAST ratings and three theoretical categories of information known to impact trust: process, purpose, and performance. Overall, our study supports the practical and theoretical viability of PADTHAI‐MM as an approach to designing trustable AI systems. [ABSTRACT FROM AUTHOR]
ISSN:07384602
DOI:10.1002/aaai.70000