DataRaceBench a benchmark suite for systematic evaluation of data race detection tools

Data races in multi-threaded parallel applications are notoriously damaging while extremely difficult to detect. Many tools have been developed to help programmers find data races. However, there is no dedicated OpenMP benchmark suite to systematically evaluate data race detection tools for their st...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:International Conference for High Performance Computing, Networking, Storage and Analysis (Online) s. 1 - 14
Hlavní autori: Liao, Chunhua, Lin, Pei-Hung, Asplund, Joshua, Schordan, Markus, Karlin, Ian
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: New York, NY, USA ACM 12.11.2017
Edícia:ACM Conferences
Predmet:
Benchmark Suite
Computing methodologies > Parallel computing methodologies > Parallel programming languages
Data Race Detection
Instruction sets
Measurement
OpenMP
Parallel programming
Runtime
Software and its engineering > Software creation and management > Software verification and validation
Software and its engineering > Software organization and properties > Software functional properties > Correctness
Static analysis
Systematics
Waste materials
data race detection
benchmark suite
openMP
ISBN:9781450351140, 145035114X
ISSN:2167-4337
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Abstract Data races in multi-threaded parallel applications are notoriously damaging while extremely difficult to detect. Many tools have been developed to help programmers find data races. However, there is no dedicated OpenMP benchmark suite to systematically evaluate data race detection tools for their strengths and limitations. In this paper, we present DataRaceBench, an open-source benchmark suite designed to systematically and quantitatively evaluate the effectiveness of data race detection tools. We focus on data race detection in programs written in OpenMP, the popular parallel programming model for multi-threaded applications. In particular, DataRaceBench includes a set of microbenchmark programs with or without data races. These microbenchmarks are either manually written, extracted from real scientific applications, or automatically generated optimization variants. We also define several metrics to represent effectiveness and efficiency of data race detection tools. Using DataRaceBench and its metrics, we evaluate four different data race detection tools: Helgrind, ThreadSanitizer, Archer, and Intel Inspector. The evaluation results show that DataRaceBench is effective to provide comparable, quantitative results and discover strengths and weaknesses of the tools being evaluated.
AbstractList Data races in multi-threaded parallel applications are notoriously damaging while extremely difficult to detect. Many tools have been developed to help programmers find data races. However, there is no dedicated OpenMP benchmark suite to systematically evaluate data race detection tools for their strengths and limitations. In this paper, we present DataRaceBench, an open-source benchmark suite designed to systematically and quantitatively evaluate the effectiveness of data race detection tools. We focus on data race detection in programs written in OpenMP, the popular parallel programming model for multi-threaded applications. In particular, DataRaceBench includes a set of microbenchmark programs with or without data races. These microbenchmarks are either manually written, extracted from real scientific applications, or automatically generated optimization variants. We also define several metrics to represent effectiveness and efficiency of data race detection tools. Using DataRaceBench and its metrics, we evaluate four different data race detection tools: Helgrind, ThreadSanitizer, Archer, and Intel Inspector. The evaluation results show that DataRaceBench is effective to provide comparable, quantitative results and discover strengths and weaknesses of the tools being evaluated.
Data races in multi-threaded parallel applications are notoriously damaging while extremely difficult to detect. Many tools have been developed to help programmers find data races. However, there is no dedicated OpenMP benchmark suite to systematically evaluate data race detection tools for their strengths and limitations. In this paper, we present DataRaceBench, an open-source benchmark suite designed to systematically and quantitatively evaluate the effectiveness of data race detection tools. We focus on data race detection in programs written in OpenMP, the popular parallel programming model for multi-threaded applications. In particular, DataRaceBench includes a set of microbenchmark programs with or without data races. These microbenchmarks are either manually written, extracted from real scientific applications, or automatically generated optimization variants. We also define several metrics to represent effectiveness and efficiency of data race detection tools. Using DataRaceBench and its metrics, we evaluate four different data race detection tools: Helgrind, ThreadSanitizer, Archer, and Intel Inspector. The evaluation results show that DataRaceBench is effective to provide comparable, quantitative results and discover strengths and weaknesses of the tools being evaluated. CCS Concepts * Software and its engineering \rightarrow Software verification and validation; Correctness; * Computing methodologies \rightarrow Parallel programming languages;
Author Asplund, Joshua
Lin, Pei-Hung
Schordan, Markus
Karlin, Ian
Liao, Chunhua
Author_xml – sequence: 1
  givenname: Chunhua
  surname: Liao
  fullname: Liao, Chunhua
  email: liao6@llnl.gov
  organization: Lawrence Livermore National Laboratory
– sequence: 2
  givenname: Pei-Hung
  surname: Lin
  fullname: Lin, Pei-Hung
  email: lin32@llnl.gov
  organization: Lawrence Livermore National Laboratory
– sequence: 3
  givenname: Joshua
  surname: Asplund
  fullname: Asplund, Joshua
  email: asplund1@llnl.gov
  organization: Lawrence Livermore National Laboratory
– sequence: 4
  givenname: Markus
  surname: Schordan
  fullname: Schordan, Markus
  email: schordan1@llnl.gov
  organization: Lawrence Livermore National Laboratory
– sequence: 5
  givenname: Ian
  surname: Karlin
  fullname: Karlin, Ian
  email: karlin1@llnl.gov
  organization: Lawrence Livermore National
BookMark eNqNj7tOxDAQRc1LYllSU_ADNAkeO35MCctTWgkJQW2N47GIYBOUbMPfE9hUVFSnOFdXOifisOs7FuIMZAVQm0sNyqL01S-N3xMFOj8Jqc3k5b5YKLCurLV2B3_csSjGsY3SSLBgarkQxQ1t6ZkavuaueTsVR5k-Ri5mLsXr3e3L6qFcP90_rq7WJWlw2zI6pwEUW5WsSTKmiLFxhr0FzBGJELxlNilldkbm7G12DLKxSIhMeinOd78tM4fPod3Q8BUQlVXGT_ZiZ6nZhNj372MAGX7aw9we5vZpWv1zGuLQctbfKh9UPQ
ContentType Conference Proceeding
Copyright 2017 ACM
Copyright_xml – notice: 2017 ACM
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1145/3126908.3126958
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList

Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 9781450351140
145035114X
EISSN 2167-4337
EndPage 14
ExternalDocumentID 9926258
Genre orig-research
GrantInformation_xml – fundername: Lawrence Livermore National Laboratory
  funderid: 10.13039/100006227
GroupedDBID 6IE
6IF
6IL
6IN
ABLEC
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
IEGSK
OCL
RIB
RIC
RIE
RIL
6IH
6IK
AAWTH
ADZIZ
CHZPO
IPLJI
ID FETCH-LOGICAL-a317t-b773112e62d65d0bdb9bc75e8619fb9aa9186ee5ddfe750ff86f7e10c69a99ea3
IEDL.DBID RIE
ISBN 9781450351140
145035114X
ISICitedReferencesCount 44
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000458161700011&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:19:14 EDT 2025
Wed Jan 31 06:44:57 EST 2024
Wed Jan 31 06:44:12 EST 2024
IsPeerReviewed false
IsScholarly false
Keywords data race detection
benchmark suite
openMP
Language English
License Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org
LinkModel DirectLink
MeetingName SC '17: The International Conference for High Performance Computing, Networking, Storage and Analysis
MergedId FETCHMERGED-LOGICAL-a317t-b773112e62d65d0bdb9bc75e8619fb9aa9186ee5ddfe750ff86f7e10c69a99ea3
PageCount 14
ParticipantIDs acm_books_10_1145_3126908_3126958
acm_books_10_1145_3126908_3126958_brief
ieee_primary_9926258
PublicationCentury 2000
PublicationDate 20171112
2017-Nov.-12
PublicationDateYYYYMMDD 2017-11-12
PublicationDate_xml – month: 11
  year: 2017
  text: 20171112
  day: 12
PublicationDecade 2010
PublicationPlace New York, NY, USA
PublicationPlace_xml – name: New York, NY, USA
PublicationSeriesTitle ACM Conferences
PublicationTitle International Conference for High Performance Computing, Networking, Storage and Analysis (Online)
PublicationTitleAbbrev SC
PublicationYear 2017
Publisher ACM
Publisher_xml – name: ACM
SSID ssib050161540
ssj0003204180
Score 1.8591539
Snippet Data races in multi-threaded parallel applications are notoriously damaging while extremely difficult to detect. Many tools have been developed to help...
SourceID ieee
acm
SourceType Publisher
StartPage 1
SubjectTerms Benchmark Suite
Computing methodologies -- Parallel computing methodologies -- Parallel programming languages
Data Race Detection
Instruction sets
Measurement
OpenMP
Parallel programming
Runtime
Software and its engineering -- Software creation and management -- Software verification and validation
Software and its engineering -- Software organization and properties -- Software functional properties -- Correctness
Static analysis
Systematics
Waste materials
Subtitle a benchmark suite for systematic evaluation of data race detection tools
Title DataRaceBench
URI https://ieeexplore.ieee.org/document/9926258
WOSCitedRecordID wos000458161700011&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFD6o-OCTlynOGxEEX-zWpk3T-KZO8WkMp7i3kssJirrK1vn7TbI6EQTxpU1DA-Uk6fm-9pzvAJykkiKzsYkyw5k7ZCZyNEhHXGlNY2mYFCoUm-D9fjEaicESnC1yYRAxBJ9hxzfDv3xT6Zn_VNYVXtyOFcuwzDmf52p9rR0WoEujW-LfwimNs6SIGzWfJGPdNKGOChadcGZBWlW__SiqEnzKzfr_nmYDtr-T88hg4XY2YQnHW7D-VZ2BNJu1BY89Wcs7qfHSXT-dkwsSGm9y8kKGM4c0icOrZLhQcibXC-FvUlniRxM_nPSwDvFaY3JfVa_TbXi4ub6_uo2aMgqRdOCgjhTnqUNVmFOTMxMro4TSnGHhuJNVQkqRFDkiM8aiww_WFrnlmMQ6F1IIlOkOrIyrMe4CcWQErZtUTH0yM7q9bXkqM5UllFpqTBuOnU1Lzw-m5TzlmZWN3cvG7m04_fOeUk2e0bah5a1evs91N8rG4Hu_d-_DGvUu14fo0QNYqSczPIRV_VE_TydHYbF8AuK8ukY
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8QwEB7WVdCTb1yfEQQvVtu0aRtvPlHURXRFbyWPCS7qVna7_n6TbF0RBPHSpqGBMkk639fOfAOwEwuKzIQ6SHTG7CHRgaVBKsikUjQUmgkufbGJrN3On574bQP2xrkwiOiDz3DfNf2_fF2qoftUdsCduB3LJ2CSJQmNRtlaX6uHefBSK5e493BMwyTKw1rPJ0rYQRxRSwbzfX9mXlxVvf0oq-K9yvns_55nDpa-0_PI7djxzEMDewsw-1WfgdTbdREeT0Ul7oTCY3v9fEiOiG-8if4LuR9arEksYiX3Yy1ncjaW_ialIW40ccPJKVY-YqtHOmX5OliCh_OzzslFUBdSCISFB1Ugsyy2uApTqlOmQ6kllypjmFv2ZCQXgkd5isi0NmgRhDF5ajKMQpVywTmKeBmavbKHK0AsHUFjpxVjl86MdnebLBaJTCJKDdW6BdvWpoVjCINilPTMitruRW33Fuz-eU8h-100LVh0Vi_eR8obRW3w1d-7t2D6onNzXVxftq_WYIY6B-wC9ug6NKv-EDdgSn1U3UF_0y-cT8Q_vY0
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=International+Conference+for+High+Performance+Computing%2C+Networking%2C+Storage+and+Analysis+%28Online%29&rft.atitle=DataRaceBench%3A+A+Benchmark+Suite+for+Systematic+Evaluation+of+Data+Race+Detection+Tools&rft.au=Liao%2C+Chunhua&rft.au=Lin%2C+Pei-Hung&rft.au=Asplund%2C+Joshua&rft.au=Schordan%2C+Markus&rft.date=2017-11-12&rft.pub=ACM&rft.eissn=2167-4337&rft.spage=1&rft.epage=14&rft_id=info:doi/10.1145%2F3126908.3126958&rft.externalDocID=9926258
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781450351140/lc.gif&client=summon&freeimage=true
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781450351140/mc.gif&client=summon&freeimage=true
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781450351140/sc.gif&client=summon&freeimage=true