Comparing the Effectiveness of Alerts and Dynamically Annotated Visualizations (DAVs) in Improving Clinical Decision Making

Objective: The aim of this study was to compare the effectiveness of two types of real-time decision support, an interrupting pop-up alert and a noninterrupting dynamically annotated visualization (DAV), in reducing clinically inappropriate diagnostic imaging orders. Background: Alerts in electronic...

Full description

Saved in:

Bibliographic Details
Published in:	Human factors Vol. 57; no. 6; pp. 1002 - 1014
Main Authors:	Rayo, Michael F., Kowalczyk, Nina, Liston, Beth W., Sanders, Elizabeth B.-N., White, Susan, Patterson, Emily S.
Format:	Journal Article
Language:	English
Published:	Los Angeles, CA SAGE Publications 01.09.2015 Human Factors and Ergonomics Society
Subjects:	Adult Computer programs Decision Making Decision making models Decision support systems Decision Support Techniques Diagnostic systems Effectiveness Electronic health records Female Human factors Human factors research Humans Imaging Male Medical Order Entry Systems - standards Patients Physicians - standards Real time Simulation Visualization Young Adult alert fatigue naturalistic decision making visualization alert decision support system alarm interruption electronic health record evidence-based medicine
ISSN:	0018-7208, 1547-8181
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	Objective: The aim of this study was to compare the effectiveness of two types of real-time decision support, an interrupting pop-up alert and a noninterrupting dynamically annotated visualization (DAV), in reducing clinically inappropriate diagnostic imaging orders. Background: Alerts in electronic health record software are frequently disregarded due to high false-alarm rates, interruptions, and uncertainty about what triggered the alert. In other settings, providing visualizations and improving understandability of the guidance has been shown to improve overall decision making. Method: Using a between-subject design, we examined the effect of two forms of decision support, alerts and DAVs, on reducing the proportion of inappropriate diagnostic imaging orders for 11 patients in a simulated environment. Nine attending and 11 resident physicians with experience using an electronic health record were randomly assigned to the form of decision support. Secondary measures were self-reported understandability, algorithm transparency, and clinical relevance. Results: Fewer inappropriate diagnostic imaging tests were ordered with DAVs than with alerts (18% vs. 34%, p < .001). The DAV was rated higher for all three secondary measures (p < .001) for all participants. Conclusion: DAVs were more effective than alerts in reducing inappropriate imaging orders and were preferred for all patient scenarios, especially scenarios where guidance was ambiguous or based on inaccurate information. Application: Creating visualizations that are permanently displayed and vary in the strength of their guidance can mitigate the risk of system performance degradation due to incomplete or incorrect data. This interaction paradigm may be applicable for other settings with high false-alarm rates or where there is a need to reduce interruptions during decision making.
AbstractList	The aim of this study was to compare the effectiveness of two types of real-time decision support, an interrupting pop-up alert and a noninterrupting dynamically annotated visualization (DAV), in reducing clinically inappropriate diagnostic imaging orders. Alerts in electronic health record software are frequently disregarded due to high false-alarm rates, interruptions, and uncertainty about what triggered the alert. In other settings, providing visualizations and improving understandability of the guidance has been shown to improve overall decision making. Using a between-subject design, we examined the effect of two forms of decision support, alerts and DAVs, on reducing the proportion of inappropriate diagnostic imaging orders for 11 patients in a simulated environment. Nine attending and 11 resident physicians with experience using an electronic health record were randomly assigned to the form of decision support. Secondary measures were self-reported understandability, algorithm transparency, and clinical relevance. Fewer inappropriate diagnostic imaging tests were ordered with DAVs than with alerts (18% vs. 34%, p < .001). The DAV was rated higher for all three secondary measures (p < .001) for all participants. DAVs were more effective than alerts in reducing inappropriate imaging orders and were preferred for all patient scenarios, especially scenarios where guidance was ambiguous or based on inaccurate information. OBJECTIVEThe aim of this study was to compare the effectiveness of two types of real-time decision support, an interrupting pop-up alert and a noninterrupting dynamically annotated visualization (DAV), in reducing clinically inappropriate diagnostic imaging orders.BACKGROUNDAlerts in electronic health record software are frequently disregarded due to high false-alarm rates, interruptions, and uncertainty about what triggered the alert. In other settings, providing visualizations and improving understandability of the guidance has been shown to improve overall decision making.METHODUsing a between-subject design, we examined the effect of two forms of decision support, alerts and DAVs, on reducing the proportion of inappropriate diagnostic imaging orders for 11 patients in a simulated environment. Nine attending and 11 resident physicians with experience using an electronic health record were randomly assigned to the form of decision support. Secondary measures were self-reported understandability, algorithm transparency, and clinical relevance.RESULTSFewer inappropriate diagnostic imaging tests were ordered with DAVs than with alerts (18% vs. 34%, p < .001). The DAV was rated higher for all three secondary measures (p < .001) for all participants.CONCLUSIONDAVs were more effective than alerts in reducing inappropriate imaging orders and were preferred for all patient scenarios, especially scenarios where guidance was ambiguous or based on inaccurate information.APPLICATIONCreating visualizations that are permanently displayed and vary in the strength of their guidance can mitigate the risk of system performance degradation due to incomplete or incorrect data. This interaction paradigm may be applicable for other settings with high false-alarm rates or where there is a need to reduce interruptions during decision making. The aim of this study was to compare the effectiveness of two types of real-time decision support, an interrupting pop-up alert and a noninterrupting dynamically annotated visualization (DAV), in reducing clinically inappropriate diagnostic imaging orders. Alerts in electronic health record software are frequently disregarded due to high false-alarm rates, interruptions, and uncertainty about what triggered the alert. In other settings, providing visualizations and improving understandability of the guidance has been shown to improve overall decision making. Using a between-subject design, we examined the effect of two forms of decision support, alerts and DAVs, on reducing the proportion of inappropriate diagnostic imaging orders for 11 patients in a simulated environment. Nine attending and 11 resident physicians with experience using an electronic health record were randomly assigned to the form of decision support. Secondary measures were self-reported understandability, algorithm transparency, and clinical relevance. Fewer inappropriate diagnostic imaging tests were ordered with DAVs than with alerts (18% vs. 34%, p < .001). The DAV was rated higher for all three secondary measures (p < .001) for all participants. DAVs were more effective than alerts in reducing inappropriate imaging orders and were preferred for all patient scenarios, especially scenarios where guidance was ambiguous or based on inaccurate information. Creating visualizations that are permanently displayed and vary in the strength of their guidance can mitigate the risk of system performance degradation due to incomplete or incorrect data. This interaction paradigm may be applicable for other settings with high false-alarm rates or where there is a need to reduce interruptions during decision making. Objective: The aim of this study was to compare the effectiveness of two types of real-time decision support, an interrupting pop-up alert and a noninterrupting dynamically annotated visualization (DAV), in reducing clinically inappropriate diagnostic imaging orders. Background: Alerts in electronic health record software are frequently disregarded due to high false-alarm rates, interruptions, and uncertainty about what triggered the alert. In other settings, providing visualizations and improving understandability of the guidance has been shown to improve overall decision making. Method: Using a between-subject design, we examined the effect of two forms of decision support, alerts and DAVs, on reducing the proportion of inappropriate diagnostic imaging orders for 11 patients in a simulated environment. Nine attending and 11 resident physicians with experience using an electronic health record were randomly assigned to the form of decision support. Secondary measures were self-reported understandability, algorithm transparency, and clinical relevance. Results: Fewer inappropriate diagnostic imaging tests were ordered with DAVs than with alerts (18% vs. 34%, p < .001). The DAV was rated higher for all three secondary measures (p < .001) for all participants. Conclusion: DAVs were more effective than alerts in reducing inappropriate imaging orders and were preferred for all patient scenarios, especially scenarios where guidance was ambiguous or based on inaccurate information. Application: Creating visualizations that are permanently displayed and vary in the strength of their guidance can mitigate the risk of system performance degradation due to incomplete or incorrect data. This interaction paradigm may be applicable for other settings with high false-alarm rates or where there is a need to reduce interruptions during decision making.
Author	Rayo, Michael F. Liston, Beth W. Patterson, Emily S. Kowalczyk, Nina Sanders, Elizabeth B.-N. White, Susan
Author_xml	– sequence: 1 givenname: Michael F. surname: Rayo fullname: Rayo, Michael F. email: mike.rayo@cogenisys.com organization: The Ohio State University, Columbus – sequence: 2 givenname: Nina surname: Kowalczyk fullname: Kowalczyk, Nina organization: The Ohio State University, Columbus – sequence: 3 givenname: Beth W. surname: Liston fullname: Liston, Beth W. organization: The Ohio State University, Columbus – sequence: 4 givenname: Elizabeth B.-N. surname: Sanders fullname: Sanders, Elizabeth B.-N. organization: The Ohio State University, Columbus – sequence: 5 givenname: Susan surname: White fullname: White, Susan organization: The Ohio State University, Columbus – sequence: 6 givenname: Emily S. surname: Patterson fullname: Patterson, Emily S. organization: The Ohio State University, Columbus
BackLink	https://www.ncbi.nlm.nih.gov/pubmed/25957043$$D View this record in MEDLINE/PubMed
BookMark	eNqNkU1v1DAURS3Uik4Le1bIEpt2kWIn8ddyNNNCpSI20G304tjFxXEG26k05c-TMKWqRgKx8uKe8_T87jE6CEMwCL2h5JxSId4TQqUoiaSMScY5f4EWlNWikFTSA7SY42LOj9BxSneEEK4q9hIdlUwxQepqgX6uhn4D0YVbnL8ZfGGt0dndm2BSwoPFS29iThhCh9fbAL3T4P0WL0MYMmTT4RuXRvDuAbIbQsKn6-VNOsMu4Kt-E4f7efDKuzB7eG20SxOGP8H3KXiFDi34ZF4_vifo6-XFl9XH4vrzh6vV8rrQlZK5qGolQFpBSmBcdMAFl6aFVnGmykrVlrCOaihNx8qOgSC2tVwZqluArra6OkGnu7nTQj9Gk3LTu6SN9xDMMKaGClYxygXl_4FSwhmvmZrQd3vo3TDGMH1kpkrJSVWKiXr7SI1tb7pmE10Pcdv8aWACyA7QcUgpGvuEUNLMJTf7JU8K31O0y7_vnyM4_y-x2IkJbs2zdf_G_wIzArY7
CODEN	HUFAA6
CitedBy_id	crossref_primary_10_1177_2327857918071001 crossref_primary_10_2345_0899_8205_51_3_214 crossref_primary_10_1093_aje_kwac034 crossref_primary_10_1007_s11301_023_00353_5 crossref_primary_10_1001_jamanetworkopen_2020_6752 crossref_primary_10_1016_j_jcjq_2017_11_008 crossref_primary_10_1177_1064804616642740 crossref_primary_10_1109_THMS_2023_3334988 crossref_primary_10_1177_2327857920091023 crossref_primary_10_2196_22013 crossref_primary_10_1016_j_jclepro_2022_132841 crossref_primary_10_1177_1541931213601877 crossref_primary_10_1177_2327857920091041 crossref_primary_10_1007_s11042_024_20497_0 crossref_primary_10_9778_cmajo_20150070 crossref_primary_10_1177_1071181320641061 crossref_primary_10_1177_0956462420906998 crossref_primary_10_1109_THMS_2022_3155714
Cites_doi	10.1016/j.ehbc.2004.05.011 10.1542/peds.2013-0864 10.1201/9781420005684 10.1136/amiajnl-2011-000049 10.1080/00140139508925269 10.1136/amiajnl-2010-000043 10.1518/001872007X230217 10.1207/s15327051hci0101_2 10.1197/jamia.M1809 10.1177/154193120004402630 10.1518/001872000779656642 10.1016/S1553-7250(13)39019-9 10.1518/hfes.46.3.497.50401 10.1201/9781315272665 10.1080/001401300421743 10.1177/1557234X0600200104 10.1518/107118109X12524442636184 10.1518/hfes.46.2.196.37335 10.1046/j.1365-2753.2001.00280.x 10.1016/j.jacr.2010.07.009 10.1109/MIS.2004.74 10.1207/s15327884mca0503_3 10.1001/archinternmed.2010.324 10.1080/00140139508925274 10.1016/S0140-6736(04)15433-0 10.1016/j.acra.2012.05.006 10.1136/amiajnl-2011-000262 10.1001/archinte.163.21.2625 10.1001/archinternmed.2009.440 10.1518/0018720024497754 10.1518/001872001775992453 10.1093/intqhc/mzq016 10.1016/B978-0-12-590241-0.50006-X 10.1148/radiol.10091228 10.1136/amiajnl-2010-000039 10.1097/01.mlr.0000199849.08389.91 10.1518/001872099779577363 10.1080/00140130701318749
ContentType	Journal Article
Copyright	2015, Human Factors and Ergonomics Society 2015, Human Factors and Ergonomics Society. Copyright Human Factors and Ergonomics Society Sep 2015
Copyright_xml	– notice: 2015, Human Factors and Ergonomics Society – notice: 2015, Human Factors and Ergonomics Society. – notice: Copyright Human Factors and Ergonomics Society Sep 2015
DBID	AAYXX CITATION CGR CUY CVF ECM EIF NPM 7QF 7QQ 7SC 7SE 7SP 7SR 7T2 7TA 7TB 7TK 7U5 8BQ 8FD C1K F28 FR3 H8D H8G JG9 JQ2 K9. KR7 L7M L~C L~D 7X8
DOI	10.1177/0018720815585666
DatabaseName	CrossRef Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed Aluminium Industry Abstracts Ceramic Abstracts Computer and Information Systems Abstracts Corrosion Abstracts Electronics & Communications Abstracts Engineered Materials Abstracts Health and Safety Science Abstracts (Full archive) Materials Business File Mechanical & Transportation Engineering Abstracts Neurosciences Abstracts Solid State and Superconductivity Abstracts METADEX Technology Research Database Environmental Sciences and Pollution Management ANTE: Abstracts in New Technology & Engineering Engineering Research Database Aerospace Database Copper Technical Reference Library Materials Research Database ProQuest Computer Science Collection ProQuest Health & Medical Complete (Alumni) Civil Engineering Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional MEDLINE - Academic
DatabaseTitle	CrossRef MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) Materials Research Database Technology Research Database Computer and Information Systems Abstracts – Academic Mechanical & Transportation Engineering Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts ProQuest Health & Medical Complete (Alumni) Materials Business File Environmental Sciences and Pollution Management Aerospace Database Copper Technical Reference Library Engineered Materials Abstracts Health & Safety Science Abstracts Advanced Technologies Database with Aerospace ANTE: Abstracts in New Technology & Engineering Civil Engineering Abstracts Aluminium Industry Abstracts Electronics & Communications Abstracts Ceramic Abstracts Neurosciences Abstracts METADEX Computer and Information Systems Abstracts Professional Solid State and Superconductivity Abstracts Engineering Research Database Corrosion Abstracts MEDLINE - Academic
DatabaseTitleList	Materials Research Database MEDLINE - Academic MEDLINE Materials Research Database
Database_xml	– sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering Psychology
EISSN	1547-8181
EndPage	1014
ExternalDocumentID	3809337791 25957043 10_1177_0018720815585666 10.1177_0018720815585666
Genre	Randomized Controlled Trial Journal Article Comparative Study Feature
GroupedDBID	--- -TM -TN -~X .2G .2L .2N .55 .DC .GJ 01A 09Z 0R~ 1~K 29I 2FS 31W 31X 31Z 36B 3EH 4.4 53G 54M 5GY 5VS 7X2 7X7 7XC 85S 88E 88I 8AF 8CJ 8FE 8FG 8FH 8FI 8FJ 8G5 8R4 8R5 AABOD AACKU AACTG AADIR AADUE AAGGD AAGLT AAJOX AAJPV AAKTJ AAMFR AANSI AAPEO AAQDB AAQXI AARIX AATAA AATBZ AAWLO AAWTL AAYJJ AAYTG ABAWP ABCCA ABCJG ABDMP ABDPE ABDWY ABEIX ABFWQ ABFXH ABGFU ABHKI ABIDT ABIVO ABJCF ABKRH ABLUO ABPNF ABQKF ABQPY ABQXT ABRHV ABUJY ABUWG ABYTW ACABN ACAEP ACDXX ACFUR ACFZE ACGBL ACGFS ACGOD ACHQT ACIWK ACJER ACLZU ACOFE ACOXC ACPRK ACROE ACRPL ACSIQ ACUAV ACUFS ACUIR ACXKE ADBBV ADDLC ADEBD ADEIA ADMHG ADNMO ADNON ADNWM ADPEE ADRRZ ADSTG ADTBJ ADTOS ADUKL ADVBO ADXHL ADYCS AEDFJ AEDXQ AEILP AENEX AEOBU AEONT AEPTA AEQLS AESMA AESZF AEUHG AEUYN AEVPJ AEWDL AEWHI AEXNY AFEET AFFNX AFKBI AFKRA AFKRG AFMOU AFQAA AFRAH AFUIA AFWMB AGDVU AGKLV AGNHF AGNWV AGQPQ AGWFA AHDMH AHHFK AHMBA AHOJL AHWHD AI. AIZZC AJEFB AJUXI AJUZI ALEEW ALFTD ALIPV ALMA_UNASSIGNED_HOLDINGS ANDLU AQUVI ARTOV ASPBG ATCPS AUTPY AUVAJ AVQMV AVWKF AYAKG AYPQM AZFZN AZQEC B8T B8Z B94 BBNVY BBRGL BCR BCU BDDNI BDZRT BEC BENPR BES BGLVJ BHPHI BKOMP BKSAR BLC BMVBW BPACV BPHCQ BVXVI BYIEH CAG CBRKF CCGJY CCPQU CEADM CFDXU COF CORYS D1J D1K DD0 DD~ DE- DG~ DH. DO- DOPDO DU5 DV7 DV8 DWQXO D~Y EBS EJD F5P FAC FAS FEDTE FHBDP FJW FYUFA GNUQQ GROUPED_SAGE_PREMIER_JOURNAL_COLLECTION GUQSH H13 HCIFZ HF~ HMCUK HVGLF HZ~ IAO ICD ICJ IEA IGS IHR IOF IPO IPY ITC J8X K.F K50 K6- KQ4 L6V LK5 LK8 LPU M0K M0T M1D M1P M2M M2O M2P M2Q M7P M7R M7S N9A O9- OHT P.B P2P PADUT PATMY PCBAR PEA PHGZM PHGZT PMKZF PQQKQ PROAC PSQYO PSYQQ PTHSS PYCSY Q1R Q2X Q7O Q7P Q7V Q7X Q82 Q83 ROL S01 S0X SAUOL SCNPE SFC SGU SGV SGZ SJFOW SPV SSDHQ TN5 U5U UAP UBX UHB UKHRP UPT VH1 WH7 X7M XH6 XOL XZL YNY ZGI ZPLXX ZPPRI ZRKOI ZXP ~02 ~32 0SE AAEJI AAPII AAYXX ABUAX ACCVC AFFHD AJGYC AJHME AJVBE AMNSR CITATION PEJEM PJZUB PPXIY PQGLB CGR CUY CVF ECM EIF M4V NPM 7QF 7QQ 7SC 7SE 7SP 7SR 7T2 7TA 7TB 7TK 7U5 8BQ 8FD C1K F28 FR3 H8D H8G JG9 JQ2 K9. KR7 L7M L~C L~D 7X8
ID	FETCH-LOGICAL-c398t-3497a8f702a567da6768ebab96592394f05d1ca2ed52d5a70fbf69e1cbaad4fc3
IEDL.DBID	AEVPJ
ISICitedReferencesCount	13
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000360904300007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0018-7208
IngestDate	Tue Sep 30 20:17:34 EDT 2025 Sun Sep 28 09:58:27 EDT 2025 Tue Oct 07 05:29:17 EDT 2025 Thu Apr 03 07:10:29 EDT 2025 Tue Nov 18 22:27:07 EST 2025 Sat Nov 29 08:09:50 EST 2025 Tue Jun 17 22:27:24 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	6
Keywords	alert fatigue naturalistic decision making visualization alert decision support system alarm interruption electronic health record evidence-based medicine
Language	English
License	2015, Human Factors and Ergonomics Society.
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c398t-3497a8f702a567da6768ebab96592394f05d1ca2ed52d5a70fbf69e1cbaad4fc3
Notes	SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 14 ObjectType-Article-2 content type line 23 ObjectType-Undefined-3 ObjectType-Article-1 ObjectType-Feature-2
PMID	25957043
PQID	1712860327
PQPubID	48481
PageCount	13
ParticipantIDs	proquest_miscellaneous_1753516716 proquest_miscellaneous_1710656459 proquest_journals_1712860327 pubmed_primary_25957043 crossref_primary_10_1177_0018720815585666 crossref_citationtrail_10_1177_0018720815585666 sage_journals_10_1177_0018720815585666
PublicationCentury	2000
PublicationDate	2015-09-01
PublicationDateYYYYMMDD	2015-09-01
PublicationDate_xml	– month: 09 year: 2015 text: 2015-09-01 day: 01
PublicationDecade	2010
PublicationPlace	Los Angeles, CA
PublicationPlace_xml	– name: Los Angeles, CA – name: United States – name: Santa Monica
PublicationTitle	Human factors
PublicationTitleAlternate	Hum Factors
PublicationYear	2015
Publisher	SAGE Publications Human Factors and Ergonomics Society
Publisher_xml	– name: SAGE Publications – name: Human Factors and Ergonomics Society
References	Bliss, Gilson, Deaton 1995; 38 Ericsson, Simon 1998; 5 van der Sijs, Aarts, Vulto, Berg 2006; 13 Holden 2001; 7 Meyer, Bitan 2002; 44 Seidling, Phansalkar, Seger, Paterno, Shaykevich, Haefeli, Bates 2011; 18 Seagull 2001; 43 Klein, Woods, Bradshaw, Hoffman, Feltovich 2004; 19 Blackmore, Mecklenburg, Kaplan 2011; 8 Schwartz 2008; 9 Georgiou, Prgomet, Markewycz, Adams, Westbrook 2011; 18 Herzog, Rieger 2004; 363 Watson, Sanderson 2004; 46 Liu, Rubin 2012; 19 Meyer 2004; 46 Cameron 2004; 8 Miller, Boro, Korman, Davoren 2011; 18 Berrington de González, Mahesh, Kim, Bhargavan, Lewis, Mettler, Land 2009; 169 Glassman, Belperio, Simon, Lanto 2006; 44 Smith, Bennett, Stone 2006; 2 Sorkin, Woods 1985; 1 Bairstow, Persaud, Mendelson, Nguyen 2010; 22 Wickens, Colcombe 2007; 49 Curry, Reed 2011; 18 Patterson, Zhang, Abbott, Gibbons, Lowry, Quinn, Ramaiah, Brick 2013; 39 Carspecken, Sharek, Longhurst, Pageler 2013; 131 Lees, Lee 2007; 50 Strom, Schinnar, Aberra, Bilker, Hennessy, Leonard, Pifer 2010; 170 Weingart, Toth, Sands, Aronson, Davis, Phillips 2003; 163 Woods 1995; 38 Vartanians, Sistrom, Weilburg, Rosenthal, Thrall 2010; 255 Guerlain, Smith, Obradovich, Rudmann, Strohm, Smith, Svirbely, Sachs 1999; 41 Russ, Zillich, McManus 2009 2009 Bliss, Dunn 2000; 43 Xiao, Vicente 2000; 42 bibr5-0018720815585666 bibr35-0018720815585666 bibr27-0018720815585666 bibr43-0018720815585666 Breznitz S. (bibr7-0018720815585666) 1984 bibr14-0018720815585666 Patterson E. S. (bibr24-0018720815585666) 2010 bibr30-0018720815585666 bibr13-0018720815585666 bibr18-0018720815585666 bibr26-0018720815585666 bibr39-0018720815585666 bibr4-0018720815585666 bibr22-0018720815585666 Schwartz D. T. (bibr29-0018720815585666) 2008; 9 bibr42-0018720815585666 bibr34-0018720815585666 bibr17-0018720815585666 bibr25-0018720815585666 bibr12-0018720815585666 bibr21-0018720815585666 bibr3-0018720815585666 bibr38-0018720815585666 bibr41-0018720815585666 bibr11-0018720815585666 bibr16-0018720815585666 bibr2-0018720815585666 bibr8-0018720815585666 bibr20-0018720815585666 Russ A. L. (bibr28-0018720815585666) 2009 bibr33-0018720815585666 bibr37-0018720815585666 bibr10-0018720815585666 bibr6-0018720815585666 bibr9-0018720815585666 bibr40-0018720815585666 bibr31-0018720815585666 bibr15-0018720815585666 bibr44-0018720815585666 bibr32-0018720815585666 bibr19-0018720815585666 bibr36-0018720815585666 bibr23-0018720815585666 bibr1-0018720815585666
References_xml	– volume: 18 start-page: 335 year: 2011 end-page: 340 article-title: The impact of computerized provider order entry systems on medical-imaging services: A systematic review publication-title: Journal of the American Medical Informatics Association – volume: 38 start-page: 2371 year: 1995 end-page: 2393 article-title: The alarm problem and directed attention in dynamic fault management publication-title: Ergonomics – volume: 18 start-page: 267 year: 2011 end-page: 270 article-title: Electronic decision support for diagnostic imaging in a primary care setting publication-title: Journal of the American Medical Informatics Association – volume: 18 start-page: i45 year: 2011 end-page: i50 article-title: Provider and pharmacist responses to warfarin drug–drug interaction alerts: A study of healthcare downstream of CPOE alerts publication-title: Journal of the American Medical Informatics Association – volume: 169 start-page: 2071 year: 2009 end-page: 2077 article-title: Projected cancer risks from computed tomographic scans performed in the United States in 2007 publication-title: Archives of Internal Medicine – volume: 13 start-page: 138 year: 2006 end-page: 147 article-title: Overriding of drug safety alerts in computerized physician order entry publication-title: Journal of the American Medical Informatics Association – volume: 44 start-page: 343 year: 2002 end-page: 353 article-title: Why better operators receive worse warnings publication-title: Human Factors – volume: 2 start-page: 74 year: 2006 end-page: 108 article-title: Representation aiding to support performance on problem-solving tasks publication-title: Reviews of Human Factors and Ergonomics – volume: 22 start-page: 194 year: 2010 end-page: 200 article-title: Reducing inappropriate diagnostic practice through education and decision support publication-title: International Journal for Quality in Health Care – volume: 44 start-page: 250 year: 2006 end-page: 256 article-title: Exposure to automated drug alerts over time: Effects on clinicians’ knowledge and perceptions publication-title: Medical Care – volume: 363 start-page: 345 year: 2004 end-page: 351 article-title: Risk of cancer from diagnostic X-rays: Estimates for the UK and 14 other countries publication-title: Lancet – volume: 49 start-page: 839 year: 2007 end-page: 850 article-title: Dual-task performance consequences of imperfect alerting associated with a cockpit display of traffic information publication-title: Human Factors – volume: 39 start-page: 129 year: 2013 end-page: 135 article-title: Enhancing electronic health record usability in pediatric patient care: A scenario-based approach publication-title: Joint Commission Journal on Quality and Patient Safety – volume: 43 start-page: 1283 year: 2000 end-page: 1300 article-title: Behavioural implications of alarm mistrust as a function of task workload publication-title: Ergonomics – volume: 9 start-page: 120 year: 2008 article-title: Counter-point: Are we really ordering too many CT scans? publication-title: Western Journal of Emergency Medicine – volume: 131 start-page: e1970 year: 2013 end-page: e1973 article-title: A clinical case of electronic health record drug alert fatigue: Consequences for patient outcome publication-title: Pediatrics – volume: 5 start-page: 178 year: 1998 end-page: 186 article-title: How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking publication-title: Mind, Culture, and Activity – volume: 46 start-page: 196 year: 2004 end-page: 294 article-title: Conceptual issues in the study of dynamic hazard warnings publication-title: Human Factors – volume: 42 start-page: 87 year: 2000 end-page: 101 article-title: A framework for epistemological analysis in empirical (laboratory and field) studies publication-title: Human Factors – volume: 8 start-page: 19 issue: 1 year: 2011 end-page: 25 article-title: Effectiveness of clinical decision support in controlling inappropriate imaging publication-title: Journal of the American College of Radiology – volume: 50 start-page: 1264 year: 2007 end-page: 1286 article-title: The influence of distraction and driving context on driver response to imperfect collision warning systems publication-title: Ergonomics – volume: 170 start-page: 1578 year: 2010 end-page: 1583 article-title: Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: A randomized controlled trial publication-title: Archives of Internal Medicine – volume: 41 start-page: 72 year: 1999 end-page: 89 article-title: Interactive critiquing as a form of decision support: An empirical evaluation publication-title: Human Factors – volume: 7 start-page: 65 year: 2001 end-page: 70 article-title: Hawthorne effects and research into professional practice publication-title: Journal of Evaluation in Clinical Practice – volume: 38 start-page: 2300 year: 1995 end-page: 2312 article-title: Human probability matching behaviour in response to alarms of varying reliability publication-title: Ergonomics – volume: 19 start-page: 91 issue: 6 year: 2004 end-page: 95 article-title: Ten challenges for making automation a team player in joint human-agent activity publication-title: Intelligent Systems, IEEE – volume: 163 start-page: 2625 year: 2003 end-page: 2631 article-title: Physicians’ decisions to override computerized drug alerts in primary care publication-title: Archives of Internal Medicine – start-page: 548 year: 2009 2009 end-page: 552 article-title: A human factors investigation of medication alerts: Barriers to prescriber decision-making and clinical workflow publication-title: AMIA Annual Symposium Proceedings – volume: 1 start-page: 49 year: 1985 end-page: 75 article-title: Systems with human monitors: A signal detection analysis publication-title: Human-Computer Interaction – volume: 19 start-page: 1094 year: 2012 end-page: 1099 article-title: The role of informatics in health care reform publication-title: Academic Radiology – volume: 255 start-page: 842 year: 2010 end-page: 849 article-title: Increasing the appropriateness of outpatient imaging: Effects of a barrier to ordering low-yield examinations publication-title: Radiology – volume: 43 start-page: 66 year: 2001 end-page: 78 article-title: Anesthesia alarms in context: An observational study publication-title: Human Factors – volume: 8 start-page: 205 year: 2004 end-page: 206 article-title: Link between diagnostic X-rays and cancer uncertain publication-title: Evidence-Based Healthcare – volume: 18 start-page: 479 year: 2011 end-page: 484 article-title: Factors influencing alert acceptance: A novel approach for predicting the success of clinical decision support publication-title: Journal of the American Medical Informatics Association – volume: 46 start-page: 497 year: 2004 end-page: 517 article-title: Sonification supports eyes-free respiratory monitoring and task time-sharing publication-title: Human Factors – ident: bibr9-0018720815585666 doi: 10.1016/j.ehbc.2004.05.011 – ident: bibr10-0018720815585666 doi: 10.1542/peds.2013-0864 – ident: bibr42-0018720815585666 doi: 10.1201/9781420005684 – volume: 9 start-page: 120 year: 2008 ident: bibr29-0018720815585666 publication-title: Western Journal of Emergency Medicine – ident: bibr41-0018720815585666 – ident: bibr11-0018720815585666 doi: 10.1136/amiajnl-2011-000049 – volume-title: Cry wolf: The psychology of false alarms year: 1984 ident: bibr7-0018720815585666 – ident: bibr6-0018720815585666 doi: 10.1080/00140139508925269 – ident: bibr13-0018720815585666 doi: 10.1136/amiajnl-2010-000043 – ident: bibr39-0018720815585666 doi: 10.1518/001872007X230217 – ident: bibr33-0018720815585666 doi: 10.1207/s15327051hci0101_2 – ident: bibr35-0018720815585666 doi: 10.1197/jamia.M1809 – ident: bibr43-0018720815585666 doi: 10.1177/154193120004402630 – ident: bibr44-0018720815585666 doi: 10.1518/001872000779656642 – ident: bibr1-0018720815585666 – ident: bibr25-0018720815585666 doi: 10.1016/S1553-7250(13)39019-9 – ident: bibr37-0018720815585666 doi: 10.1518/hfes.46.3.497.50401 – ident: bibr8-0018720815585666 doi: 10.1201/9781315272665 – volume-title: Macrocognition metrics and scenarios year: 2010 ident: bibr24-0018720815585666 – ident: bibr5-0018720815585666 doi: 10.1080/001401300421743 – ident: bibr32-0018720815585666 doi: 10.1177/1557234X0600200104 – ident: bibr27-0018720815585666 doi: 10.1518/107118109X12524442636184 – ident: bibr21-0018720815585666 doi: 10.1518/hfes.46.2.196.37335 – ident: bibr17-0018720815585666 doi: 10.1046/j.1365-2753.2001.00280.x – ident: bibr4-0018720815585666 doi: 10.1016/j.jacr.2010.07.009 – ident: bibr18-0018720815585666 doi: 10.1109/MIS.2004.74 – ident: bibr12-0018720815585666 doi: 10.1207/s15327884mca0503_3 – ident: bibr34-0018720815585666 doi: 10.1001/archinternmed.2010.324 – ident: bibr40-0018720815585666 doi: 10.1080/00140139508925274 – ident: bibr16-0018720815585666 doi: 10.1016/S0140-6736(04)15433-0 – ident: bibr20-0018720815585666 doi: 10.1016/j.acra.2012.05.006 – ident: bibr23-0018720815585666 doi: 10.1136/amiajnl-2011-000262 – ident: bibr38-0018720815585666 doi: 10.1001/archinte.163.21.2625 – ident: bibr3-0018720815585666 doi: 10.1001/archinternmed.2009.440 – ident: bibr22-0018720815585666 doi: 10.1518/0018720024497754 – ident: bibr30-0018720815585666 doi: 10.1518/001872001775992453 – ident: bibr2-0018720815585666 doi: 10.1093/intqhc/mzq016 – ident: bibr26-0018720815585666 doi: 10.1016/B978-0-12-590241-0.50006-X – ident: bibr36-0018720815585666 doi: 10.1148/radiol.10091228 – start-page: 548 year: 2009 ident: bibr28-0018720815585666 publication-title: AMIA Annual Symposium Proceedings – ident: bibr31-0018720815585666 doi: 10.1136/amiajnl-2010-000039 – ident: bibr14-0018720815585666 doi: 10.1097/01.mlr.0000199849.08389.91 – ident: bibr15-0018720815585666 doi: 10.1518/001872099779577363 – ident: bibr19-0018720815585666 doi: 10.1080/00140130701318749
SSID	ssj0006935
Score	2.185766
Snippet	Objective: The aim of this study was to compare the effectiveness of two types of real-time decision support, an interrupting pop-up alert and a... The aim of this study was to compare the effectiveness of two types of real-time decision support, an interrupting pop-up alert and a noninterrupting... OBJECTIVEThe aim of this study was to compare the effectiveness of two types of real-time decision support, an interrupting pop-up alert and a noninterrupting...
SourceID	proquest pubmed crossref sage
SourceType	Aggregation Database Index Database Enrichment Source Publisher
StartPage	1002
SubjectTerms	Adult Computer programs Decision Making Decision making models Decision support systems Decision Support Techniques Diagnostic systems Effectiveness Electronic health records Female Human factors Human factors research Humans Imaging Male Medical Order Entry Systems - standards Patients Physicians - standards Real time Simulation Visualization Young Adult
Title	Comparing the Effectiveness of Alerts and Dynamically Annotated Visualizations (DAVs) in Improving Clinical Decision Making
URI	https://journals.sagepub.com/doi/full/10.1177/0018720815585666 https://www.ncbi.nlm.nih.gov/pubmed/25957043 https://www.proquest.com/docview/1712860327 https://www.proquest.com/docview/1710656459 https://www.proquest.com/docview/1753516716
Volume	57
WOSCitedRecordID	wos000360904300007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVSPB databaseName: SAGE Journals HSS Package 2015 customDbUrl: eissn: 1547-8181 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0006935 issn: 0018-7208 databaseCode: AEVPJ dateStart: 19990301 isFulltext: true titleUrlDefault: http://journals.sagepub.com/ providerName: SAGE Publications
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwvV1La9wwEB6STQ_poU3Tl_NCgVKag7K2bFnyKZhsQigk5NAuezOyLJeFjbasvYWlf76SLbt5kJBTrraMZHs080kz-j6AL0SWNqwozLmkOJIRxzkTPs5jRaQgJCdJKzbBrq74ZJJcr4HuzsK4L1gd27IqM6LGWdvZbXejhy7JOGyk5IiJZtSgXQPAT5b1TdbudneiGvaKTU8vb2xmW9p6yBXuTretw4aB4gEdwEZ6Nr7-3vvuOAmd5gHHtoP_ic0Hfd4NZA_Q6Z3KsCZYnb996dfcgjcO1qK0tcN3sKb0Nry-RXa4DZu9r129h7-nrf6h_oUMAkUth7JzvGheonSmFnWFhC7QaKVFw2kwW6FU67lFxwUaTyt7HNQdIkXfRum4OkJTjfpdEuQoT2do5HSE0GUjvfUBfp6f_Ti9wE4HAssw4TUOo4QJXjKfCBqzQsRmiaRykVsuRKvsXvq0CIxhqYKSggrml3kZJyqQuRBFVMrwIwz0XKvPgJKYE5kb2GlWnVEoAq4YLRqF0KQ0rsj3YNj91Ew6knSr1THLgo4X_d4_8eCof-J3SxDyRNu9zk6yzgIyY4eEx35ImAeH_W0zyW3mRmg1XzZtDFS0vD9PtaEhDWKz_vXgU2uD_YDMGpcyPwo9-GoN7lbnj4x057kNd2HTzHXaVt3twaBeLNU-vJJ_6mm1OIB1NuEHbpL9A-f6Nz8
linkProvider	SAGE Publications
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1RT9UwFD7Bi4nwIIooQ5SaGCMPw61b1-5x4UIQ4YYHvOFt6bqO3GTpNXcXE-Kf53TrnReJxMTnnW5NT885X9f2-wA-UlXZsqJ9IRTzYxULv-Ay8ItEUyUpLWjaiU3w0UhcXaUXS1JfbgSbA3usCnvUJus-ui1TklWRo1jIGAJdxN5PYDVG1B4MYDU7Gl-c9mk4SSMnXyB82-D3HuWDd9yvSQ-A5r1DXm3dOd74jx6_gOcObJKsmx0vYUWbTVhfoiDchLU-A96-gl-HnSqhuSaIC0nHbOzSIZlWJKv1bN4QaUoy7JTsZV3fksyYqcWsJRlPGntJ013tJJ-H2bjZJxND-n8XxBGR1mTo1H3IeSuItQXfj48uD098p87gqygVcz-KUy5FxQMqWcJLmeDCRReysAyFVm-9ClgZort1yWjJJA-qokpSHapCyjKuVPQaBmZq9DaQNBFUFQgGcS0YRzIUmrOy1e1MK0wQgQdfFv7JlaMutwoadR4u2Mr_GGMP9vsWPzrajkdsdxcuzxfOzEOOhTsJIso9-NA_xtCz-ynS6OlNa4MAzrLxPGbDIhYmuCr14E03nfoO4cqT8SCOPPhk587Sx__S051_NdyDZyeX52f52dfRt7ewhtHIunNxuzCYz270O3iqfs4nzey9i5o7TeYOSQ
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3Nb9MwFH8aHULjwGAMyBhgJITYIWvixLFzjNZVbEDVA1S7RY4_pkqROzUd0sQ_Pztxw8bEhMQ5z4nl5_cVP_9-AB-w0C6sqJAxQcJUpCysKI_CKlNYcIwrnHdkE3QyYWdn-dT35ri7MH4Fm0PXVmVn1DprZ90XUg_9GeOwZZLDNpgRm-za_PsBbNqqJsID2CyOZ9PT3hVneeIpDFjoBvw-p7zzjttx6U6yeavRq4094-3_nPVTeOKTTlR0u-QZbCizA49vQBHuwFbvCa-ew6-jjp3QnCObH6IO4di7RbTQqKjVctUgbiQadYz2vK6vUGHMwuWuEs3mjbus6a94ok-jYtYcoLlB_T8M5AFJazTyLD_oW0uMtQs_xsffjz6HnqUhFEnOVmGS5pQzTSPMSUYlz2wBoypeOaRCx7uuIyJjq3YlCZaE00hXOstVLCrOZapF8gIGZmHUK0B5xrCobFJoa8I04TFTlMiWvzPX1lFEAQzXOiqFhzB3TBp1Ga9Ry_9Y4wAO-hEXHXzHPbL7a7WXa4WWMbUBPIsSTAN43z-2JujOVbhRi8tWxiZyDpXnPhmSkDiz1WkAL7st1U_IVqCERmkSwEe3f258_C8z3ftXwXfwaDoal19PJl9ew5Y1StK1x-3DYLW8VG_gofi5mjfLt95wrgExUBC9
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Comparing+the+Effectiveness+of+Alerts+and+Dynamically+Annotated+Visualizations+%28DAVs%29+in+Improving+Clinical+Decision+Making&rft.jtitle=Human+factors&rft.au=Rayo%2C+Michael+F&rft.au=Kowalczyk%2C+Nina&rft.au=Liston%2C+Beth+W&rft.au=Sanders%2C+Elizabeth+B-N&rft.date=2015-09-01&rft.issn=0018-7208&rft.volume=57&rft.issue=6&rft.spage=1002&rft.epage=1014&rft_id=info:doi/10.1177%2F0018720815585666&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-7208&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-7208&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-7208&client=summon