Data Quality Matters: A Case Study on Data Label Correctness for Security Bug Report Prediction

In the research of mining software repositories, we need to label a large amount of data to construct a predictive model. The correctness of the labels will affect the performance of a model substantially. However, limited studies have been performed to investigate the impact of mislabeled instances...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE transactions on software engineering Jg. 48; H. 7; S. 2541 - 2556
Hauptverfasser:	Wu, Xiaoxue, Zheng, Wei, Xia, Xin, Lo, David
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	New York IEEE 01.07.2022 IEEE Computer Society
Schlagworte:	Case studies Chromium Classification Computer bugs Cybersecurity Data models data quality Datasets Debugging label correctness Noise measurement Prediction models Predictive models Security Security bug report prediction Tuning
ISSN:	0098-5589, 1939-3520
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	In the research of mining software repositories, we need to label a large amount of data to construct a predictive model. The correctness of the labels will affect the performance of a model substantially. However, limited studies have been performed to investigate the impact of mislabeled instances on a predictive model. To bridge the gap, in this article, we perform a case study on the security bug report (SBR) prediction. We found five publicly available datasets for SBR prediction contains many mislabeled instances, which lead to the poor performance of SBR prediction models of recent studies (e.g., the work of Peters et al. and Shu et al. ). Furthermore, it might mislead the research direction of SBR prediction. In this article, we first improve the label correctness of these five datasets by manually analyzing each bug report, and we find 749 SBRs, which are originally mislabeled as Non-SBRs (NSBRs). We then evaluate the impacts of datasets label correctness by comparing the performance of the classification models on both the noisy (i.e., before our correction) and the clean (i.e., after our correction) datasets. The results show that the cleaned datasets result in improvement in the performance of classification models. The performance of the approaches proposed by Peters et al. and Shu et al. on the clean datasets is much better than on the noisy datasets. Furthermore, with the clean datasets, the simple text classification models could significantly outperform the security keywords-matrix-based approaches applied by Peters et al. and Shu et al.
AbstractList	In the research of mining software repositories, we need to label a large amount of data to construct a predictive model. The correctness of the labels will affect the performance of a model substantially. However, limited studies have been performed to investigate the impact of mislabeled instances on a predictive model. To bridge the gap, in this article, we perform a case study on the security bug report (SBR) prediction. We found five publicly available datasets for SBR prediction contains many mislabeled instances, which lead to the poor performance of SBR prediction models of recent studies (e.g., the work of Peters et al. and Shu et al. ). Furthermore, it might mislead the research direction of SBR prediction. In this article, we first improve the label correctness of these five datasets by manually analyzing each bug report, and we find 749 SBRs, which are originally mislabeled as Non-SBRs (NSBRs). We then evaluate the impacts of datasets label correctness by comparing the performance of the classification models on both the noisy (i.e., before our correction) and the clean (i.e., after our correction) datasets. The results show that the cleaned datasets result in improvement in the performance of classification models. The performance of the approaches proposed by Peters et al. and Shu et al. on the clean datasets is much better than on the noisy datasets. Furthermore, with the clean datasets, the simple text classification models could significantly outperform the security keywords-matrix-based approaches applied by Peters et al. and Shu et al.
Author	Lo, David Wu, Xiaoxue Zheng, Wei Xia, Xin
Author_xml	– sequence: 1 givenname: Xiaoxue orcidid: 0000-0002-7567-3643 surname: Wu fullname: Wu, Xiaoxue email: wuxiaoxue00@gmail.com organization: School of Cyberspace Security, Northwestern Polytechnical University, Xi'an, Shaanxi, China – sequence: 2 givenname: Wei orcidid: 0000-0001-7969-1630 surname: Zheng fullname: Zheng, Wei email: wzheng@nwpu.edu.cn organization: School of Software, Northwestern Polytechnical University, Xi'an, Shaanxi, China – sequence: 3 givenname: Xin orcidid: 0000-0002-6302-3256 surname: Xia fullname: Xia, Xin email: xin.xia@monash.edu organization: Faculty of Information Technology, Monash University, Melbourne, Victoria, Australia – sequence: 4 givenname: David orcidid: 0000-0002-4367-7201 surname: Lo fullname: Lo, David email: davidlo@smu.edu.sg organization: School of Information Systems, Singapore Management University, Singapore
BookMark	eNp9kDtPwzAUhS1UJEphR2KxxJxyHcexzQalPKQiHoXZcpwbFFTiYjtD_z0trRgYmO5yvnN0v0My6HyHhJwwGDMG-vx1Ph3nkLMxh5LLXO6RIdNcZ1zkMCBDAK0yIZQ-IIcxfgCAkFIMibm2ydLn3i7atKIPNiUM8YJe0omNSOepr1fUd_QnNbMVLujEh4AudRgjbXygc3R92MBX_Tt9waUPiT4FrFuXWt8dkf3GLiIe7-6IvN1MXyd32ezx9n5yOcscFzJlORfolOSqEK6sQGlRV1AJy1FDUbCmKpTjqFhdKFbmGhq0Wle6Zo3ltZKSj8jZtncZ_FePMZkP34duPWnyUmkFsixgnYJtygUfY8DGLEP7acPKMDAbjWat0Ww0mp3GNVL-QVyb7Oa1FGy7-A883YItIv7uaC4Z15x_A7BkgCU
CODEN	IESEDJ
CitedBy_id	crossref_primary_10_3390_risks12110172 crossref_primary_10_3390_app12178906 crossref_primary_10_1155_2023_7941878 crossref_primary_10_3390_electronics11193164 crossref_primary_10_1155_2022_2133712 crossref_primary_10_1155_2022_7419736 crossref_primary_10_3390_su142114385 crossref_primary_10_1155_2022_9620555 crossref_primary_10_1007_s11063_023_11173_9 crossref_primary_10_1093_jcde_qwac119 crossref_primary_10_3390_jrfm17120577 crossref_primary_10_1016_j_infsof_2023_107354 crossref_primary_10_1007_s10207_023_00793_w crossref_primary_10_1016_j_jss_2023_111838 crossref_primary_10_1016_j_infsof_2023_107350 crossref_primary_10_1093_jcde_qwac075 crossref_primary_10_1109_TEM_2022_3207326 crossref_primary_10_1007_s13726_024_01434_2 crossref_primary_10_1093_jcde_qwac111 crossref_primary_10_1080_17455030_2022_2164377 crossref_primary_10_1109_TR_2021_3118026 crossref_primary_10_1016_j_eswa_2024_125107 crossref_primary_10_3389_fcomp_2024_1428013 crossref_primary_10_1007_s11276_022_03177_5 crossref_primary_10_3390_electronics11172773 crossref_primary_10_3390_buildings14041120 crossref_primary_10_1002_widm_1520 crossref_primary_10_1007_s10515_024_00435_y crossref_primary_10_1145_3715778 crossref_primary_10_1155_2022_1008617 crossref_primary_10_1017_jmo_2022_74 crossref_primary_10_1016_j_ygeno_2025_111036 crossref_primary_10_1017_jmo_2022_77 crossref_primary_10_1109_TR_2024_3455390 crossref_primary_10_3390_math12162469 crossref_primary_10_1016_j_neucom_2022_12_048 crossref_primary_10_1155_2022_8217774 crossref_primary_10_1016_j_knosys_2023_111258 crossref_primary_10_1007_s11334_024_00584_z crossref_primary_10_1109_ACCESS_2022_3221454 crossref_primary_10_1038_s41598_024_82592_4 crossref_primary_10_1007_s10664_023_10418_4 crossref_primary_10_1093_jcde_qwac085 crossref_primary_10_1016_j_infsof_2025_107778 crossref_primary_10_1109_ACCESS_2025_3564818 crossref_primary_10_1007_s11432_022_3803_9 crossref_primary_10_4018_JGIM_302642 crossref_primary_10_1007_s11356_022_24471_x crossref_primary_10_1145_3750040 crossref_primary_10_3390_e24121783 crossref_primary_10_1155_2022_8481452 crossref_primary_10_1002_smr_70044 crossref_primary_10_1007_s12517_021_09005_y crossref_primary_10_1016_j_scico_2024_103078 crossref_primary_10_1109_TSE_2024_3364675
Cites_doi	10.1037/h0031619 10.1109/TSE.2017.2787653 10.1109/ICSE.2015.64 10.1109/MSR.2010.5463340 10.1145/3106237.3106256 10.1145/3338906.3338947 10.1023/A:I0I0933404324 10.1145/3106237.3106285 10.1145/185403.185412 10.1109/MSP.2004.84 10.1145/1321631.1321691 10.1007/s11390-017-1713-3 10.1145/3297858.3304013 10.1002/eap.2043 10.1109/COMPSAC.2014.17 10.1109/MSR.2015.78 10.1145/1370788.1370807 10.1007/978-3-319-72308-2_9 10.1145/3219819.3219873 10.1109/ICPC.2019.00021 10.1109/ISSRE.2016.29 10.1109/ESEM.2017.24 10.1109/ICSE.2015.91 10.1109/TSE.2019.2929761 10.1145/2724721 10.1109/TSE.2018.2877678 10.1109/TSE.2018.2864217 10.1007/s10515-014-0162-2 10.1109/ASE.2006.23 10.1109/ICECCS.2014.25 10.1109/SERE.2013.31 10.1145/1082983.1083147 10.1109/TEVC.2010.2059031 10.1145/3238147.3238204 10.1145/3324916 10.1109/ICSME.2016.62 10.1007/s10664-015-9409-1 10.1109/ASE.2019.00026 10.1145/3238147.3238190 10.1109/ICSM.2015.7332459 10.1007/978-3-030-05318-5_1 10.1109/ICSE.2013.6606585 10.1016/j.infsof.2018.11.005 10.1109/TSE.2013.11 10.1109/MICRO.2010.51 10.1109/TSE.2018.2887384 10.1145/3503509 10.1109/TSE.2019.2924006 10.1145/2992786 10.1145/1985793.1985859 10.1109/ICSME.2014.77 10.1145/2884781.2884877 10.1109/HSI.2012.22 10.1109/TSE.2017.2754374 10.1109/TSE.2018.2876537 10.1145/3338906.3338941 10.1145/1985441.1985457 10.1109/ICSE.2015.93 10.1109/ICROIT.2014.6798341
ContentType	Journal Article
Copyright	Copyright IEEE Computer Society 2022
Copyright_xml	– notice: Copyright IEEE Computer Society 2022
DBID	97E RIA RIE AAYXX CITATION JQ2 K9.
DOI	10.1109/TSE.2021.3063727
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef ProQuest Computer Science Collection ProQuest Health & Medical Complete (Alumni)
DatabaseTitle	CrossRef ProQuest Health & Medical Complete (Alumni) ProQuest Computer Science Collection
DatabaseTitleList	ProQuest Health & Medical Complete (Alumni)
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Computer Science
EISSN	1939-3520
EndPage	2556
ExternalDocumentID	10_1109_TSE_2021_3063727 9371393
Genre	orig-research
GrantInformation_xml	– fundername: Key Laboratory of Advanced Perception and Intelligent Control of High-end Equipment, Ministry of Education grantid: GDSC202006 – fundername: Innovation Foundation for Doctor Dissertation of Northwestern Polytechnical University grantid: CX202067
GroupedDBID	--Z -DZ -~X .DC 0R~ 29I 4.4 5GY 6IK 85S 8R4 8R5 97E AAJGR AARMG AASAJ AAWTH ABAZT ABPPZ ABQJQ ABVLG ACGFO ACGOD ACIWK ACNCT AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BKOMP BPEOZ CS3 DU5 EBS EDO EJD HZ~ I-F IEDLZ IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P Q2X RIA RIE RNS RXW S10 TAE TN5 TWZ UHB UPT WH7 YZZ AAYXX CITATION JQ2 K9.
ID	FETCH-LOGICAL-c357t-235ec873845c6b0895db0b5a3e90441fb48c3e81d4816290fea99b9d1fa3d8773
IEDL.DBID	RIE
ISICitedReferencesCount	107
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000825973800002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0098-5589
IngestDate	Fri Oct 03 04:01:14 EDT 2025 Sat Nov 29 03:10:26 EST 2025 Tue Nov 18 22:18:27 EST 2025 Wed Aug 27 02:25:43 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	7
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c357t-235ec873845c6b0895db0b5a3e90441fb48c3e81d4816290fea99b9d1fa3d8773
Notes	ObjectType-Case Study-2 SourceType-Scholarly Journals-1 content type line 14 ObjectType-Feature-4 ObjectType-Report-1 ObjectType-Article-3
ORCID	0000-0001-7969-1630 0000-0002-7567-3643 0000-0002-6302-3256 0000-0002-4367-7201
PQID	2689807640
PQPubID	21418
PageCount	16
ParticipantIDs	crossref_citationtrail_10_1109_TSE_2021_3063727 ieee_primary_9371393 proquest_journals_2689807640 crossref_primary_10_1109_TSE_2021_3063727
PublicationCentury	2000
PublicationDate	2022-07-01
PublicationDateYYYYMMDD	2022-07-01
PublicationDate_xml	– month: 07 year: 2022 text: 2022-07-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on software engineering
PublicationTitleAbbrev	TSE
PublicationYear	2022
Publisher	IEEE IEEE Computer Society
Publisher_xml	– name: IEEE – name: IEEE Computer Society
References	ref13 ref15 ref59 ref14 ref58 ref52 ref11 ref55 ref10 ref54 ref16 Jack (ref56) 2007 ref19 (ref46) 2018 Thomas (ref17) ref47 ref41 ref44 Mandt (ref57) 2011 ref43 (ref50) 2020 Shu (ref12) 2019 ref49 ref8 (ref45) 2020 ref7 (ref22) 2019 (ref1) 2020 ref9 ref4 ref3 ref6 ref5 ref40 ref35 (ref48) 2020 ref34 (ref51) 2020 ref36 ref31 ref75 ref30 ref74 ref33 ref32 ref76 Liu (ref42) ref2 ref39 Spencer (ref53) 2009 ref38 (ref37) 2004 ref71 (ref18) 2011 ref70 ref73 ref72 ref24 ref68 ref23 ref67 ref26 ref25 ref69 ref20 ref64 Arnold (ref29) ref63 ref66 ref21 ref65 (ref60) 2020 ref28 ref27 ref62 ref61
References_xml	– start-page: 421 volume-title: Proc. 3rd Int. Conf. Softw. Testing Verification Validation ident: ref17 article-title: Searching for a needle in a haystack: Predicting security vulnerabilities for windows vista – ident: ref55 doi: 10.1037/h0031619 – ident: ref11 doi: 10.1109/TSE.2017.2787653 – ident: ref20 doi: 10.1109/ICSE.2015.64 – year: 2020 ident: ref45 article-title: CVE – ident: ref13 doi: 10.1109/MSR.2010.5463340 – year: 2019 ident: ref22 article-title: 2019 TOP 25 CWEs – ident: ref23 doi: 10.1145/3106237.3106256 – ident: ref25 doi: 10.1145/3338906.3338947 – ident: ref63 doi: 10.1023/A:I0I0933404324 – ident: ref26 doi: 10.1145/3106237.3106285 – ident: ref49 doi: 10.1145/185403.185412 – ident: ref47 doi: 10.1109/MSP.2004.84 – ident: ref43 doi: 10.1145/1321631.1321691 – ident: ref27 doi: 10.1007/s11390-017-1713-3 – volume-title: Card Sorting: Designing Usable Categories year: 2009 ident: ref53 – ident: ref41 doi: 10.1145/3297858.3304013 – ident: ref39 doi: 10.1002/eap.2043 – ident: ref34 doi: 10.1109/COMPSAC.2014.17 – year: 2020 ident: ref51 article-title: CWE 1228 – year: 2007 ident: ref56 article-title: Vector rewrite attack: Exploitable null pointer vulnerabilities on arm and Xscale architectures publication-title: White Paper, Juniper Networks – ident: ref19 doi: 10.1109/MSR.2015.78 – ident: ref36 doi: 10.1145/1370788.1370807 – year: 2018 ident: ref46 article-title: JIRA – ident: ref58 doi: 10.1007/978-3-319-72308-2_9 – ident: ref65 doi: 10.1145/3219819.3219873 – volume-title: Proc. 12th Workshop Hot Top. Operating Syst. ident: ref29 article-title: Security impact ratings considered harmful – ident: ref73 doi: 10.1109/ICPC.2019.00021 – ident: ref76 doi: 10.1109/ISSRE.2016.29 – ident: ref44 doi: 10.1109/ESEM.2017.24 – year: 2019 ident: ref12 article-title: Better security bug report classification via hyperparameter optimization – year: 2020 ident: ref50 article-title: Software CWEs – ident: ref7 doi: 10.1109/ICSE.2015.91 – ident: ref31 doi: 10.1109/TSE.2019.2929761 – ident: ref9 doi: 10.1145/2724721 – ident: ref54 doi: 10.1109/TSE.2018.2877678 – ident: ref74 doi: 10.1109/TSE.2018.2864217 – year: 2020 ident: ref60 article-title: Scikit learn – ident: ref67 doi: 10.1007/s10515-014-0162-2 – ident: ref30 doi: 10.1109/ASE.2006.23 – year: 2011 ident: ref57 article-title: Kernel attacks through user-mode callbacks publication-title: BlackHat USA – ident: ref32 doi: 10.1109/ICECCS.2014.25 – ident: ref16 doi: 10.1109/SERE.2013.31 – year: 2011 ident: ref18 article-title: Mining software repository challenge – ident: ref35 doi: 10.1145/1082983.1083147 – ident: ref59 doi: 10.1109/TEVC.2010.2059031 – ident: ref75 doi: 10.1145/3238147.3238204 – year: 2020 ident: ref48 article-title: CWE – ident: ref61 doi: 10.1145/3324916 – ident: ref72 doi: 10.1109/ICSME.2016.62 – ident: ref3 doi: 10.1007/s10664-015-9409-1 – ident: ref10 doi: 10.1109/ASE.2019.00026 – ident: ref24 doi: 10.1145/3238147.3238190 – ident: ref70 doi: 10.1109/ICSM.2015.7332459 – ident: ref38 doi: 10.1007/978-3-030-05318-5_1 – ident: ref33 doi: 10.1109/ICSE.2013.6606585 – ident: ref62 doi: 10.1016/j.infsof.2018.11.005 – year: 2020 ident: ref1 article-title: Mining software repository – ident: ref2 doi: 10.1109/TSE.2013.11 – ident: ref21 doi: 10.1109/MICRO.2010.51 – ident: ref40 doi: 10.1109/TSE.2018.2887384 – ident: ref64 doi: 10.1145/3503509 – ident: ref52 doi: 10.1109/TSE.2019.2924006 – ident: ref8 doi: 10.1145/2992786 – ident: ref5 doi: 10.1145/1985793.1985859 – ident: ref69 doi: 10.1109/ICSME.2014.77 – ident: ref71 doi: 10.1145/2884781.2884877 – ident: ref28 doi: 10.1109/HSI.2012.22 – start-page: 1547 volume-title: Proc. 42nd Int. Conf. Softw. Eng. ident: ref42 article-title: A large-scale empirical study on vulnerability distribution within projects and the lessons learned – ident: ref68 doi: 10.1109/TSE.2017.2754374 – ident: ref66 doi: 10.1109/TSE.2018.2876537 – ident: ref6 doi: 10.1145/3338906.3338941 – ident: ref15 doi: 10.1145/1985441.1985457 – volume-title: Hackers and Painters: Big Ideas From the Computer Age year: 2004 ident: ref37 – ident: ref4 doi: 10.1109/ICSE.2015.93 – ident: ref14 doi: 10.1109/ICROIT.2014.6798341
SSID	ssj0005775 ssib053395008
Score	2.654709
Snippet	In the research of mining software repositories, we need to label a large amount of data to construct a predictive model. The correctness of the labels will...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	2541
SubjectTerms	Case studies Chromium Classification Computer bugs Cybersecurity Data models data quality Datasets Debugging label correctness Noise measurement Prediction models Predictive models Security Security bug report prediction Tuning
Title	Data Quality Matters: A Case Study on Data Label Correctness for Security Bug Report Prediction
URI	https://ieeexplore.ieee.org/document/9371393 https://www.proquest.com/docview/2689807640
Volume	48
WOSCitedRecordID	wos000825973800002&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1939-3520 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: RIE dateStart: 19750101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFH7M4cGLv6Y4nZKDF8G6tGmaxJtOhwcdgyl4C0mTiTA22TrB_94kbaegCN56SCD0a_K-1_fyfQCnjFFqE51FJmXGJSiKR5oI7bJW49lybKjFwWyCDQb8-VkMG3C-ugtjrQ3NZ_bCP4ZavpnlS_-rrOu124gga7DGWFbe1fpq52CM1vqYlHJRlySx6D6Obl0imMQXjh4T5v1jvoWg4Kny4yAO0aW_9b91bcNmxSLRVQn7DjTsdBe2aocGVG3YFsgbVShU6mR8oIdSS_MSXaGeC17I9xB-oNkUhVH3StsJ6nm3jrzwByByfBaNKn87dL18QSVbR8O5r-54RPfgqX_72LuLKkuFKCeUFVFCqM05IzyleaYxF9RorKkiVmBHjMY65TmxjsOmPM4SgcdWCaGFiceKGM4Y2YfmdDa1B4CwOxhpgjNfXkljo7jmIlXC69VxF_hVG7r1W5Z5pTfubS8mMuQdWEiHi_S4yAqXNpytZryVWht_jG15HFbjKgja0KmBlNVmXMgk44JjlqX48PdZR7CR-FsNoQu3A81ivrTHsJ6_F6-L-Un4zj4BWEXNkw
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3Pa9swFH6kaaG7rD_H0nWdDrsM6ka2JEvarUtbOpqEQFLoTUiWUgYlKalTyH9fSbazQstgNx8kEP4sve_5PX0fwHfOGXOZyRNLufUJihaJIdL4rNUGtpxa5nA0m-DDobi7k6MWnK7vwjjnYvOZOwuPsZZv58Uy_CrrBu02IskGbDJKM1zd1vrb0ME5axQyGROyKUpi2Z2ML30qmKVnniATHhxkXgWh6Kry5iiO8eVq5_9Wtgsfax6Jzivg96DlZvuw03g0oHrLHoC60KVGlVLGCg0qNc2f6Bz1fPhCoYtwheYzFEf1tXEPqBf8OooyHIHIM1o0rh3u0K_lPar4OhotQn0nYHoIt1eXk951UpsqJAVhvEwywlwhOBGUFbnBQjJrsGGaOIk9NZoaKgriPIulIs0ziadOS2mkTaeaWME5-QTt2XzmPgPC_mhkGc5DgYWmVgsjJNUyKNYJH_p1B7rNW1ZFrTgejC8eVMw8sFQeFxVwUTUuHfixnvFYqW38Y-xBwGE9roagA8cNkKrejk8qy4UUmOcUH70_6xtsX08GfdX_Pbz5Ah-ycMch9uQeQ7tcLN1X2Cqeyz9Pi5P4zb0ANQjQ2g
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Data+Quality+Matters%3A+A+Case+Study+on+Data+Label+Correctness+for+Security+Bug+Report+Prediction&rft.jtitle=IEEE+transactions+on+software+engineering&rft.au=Wu%2C+Xiaoxue&rft.au=Zheng%2C+Wei&rft.au=Xia%2C+Xin&rft.au=Lo%2C+David&rft.date=2022-07-01&rft.pub=IEEE+Computer+Society&rft.issn=0098-5589&rft.eissn=1939-3520&rft.volume=48&rft.issue=7&rft.spage=2541&rft_id=info:doi/10.1109%2FTSE.2021.3063727&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0098-5589&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0098-5589&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0098-5589&client=summon