A ground-truth dataset and classification model for detecting bots in GitHub issue and PR comments

Bots are frequently used in Github repositories to automate repetitive activities that are part of the distributed software development process. They communicate with human actors through comments. While detecting their presence is important for many reasons, no large and representative ground-truth...

Full description

Saved in:

Bibliographic Details
Published in:	The Journal of systems and software Vol. 175; p. 110911
Main Authors:	Golzadeh, Mehdi, Decan, Alexandre, Legay, Damien, Mens, Tom
Format:	Journal Article
Language:	English
Published:	Elsevier Inc 01.05.2021
Subjects:	Bot identification Classification model Distributed software development GitHub repositories Text similarity Bot identification Distributed software development Classification model Text similarity GitHub repositories
ISSN:	0164-1212, 1873-1228
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	Bots are frequently used in Github repositories to automate repetitive activities that are part of the distributed software development process. They communicate with human actors through comments. While detecting their presence is important for many reasons, no large and representative ground-truth dataset is available, nor are classification models to detect and validate bots on the basis of such a dataset. This paper proposes a ground-truth dataset, based on a manual analysis with high interrater agreement, of pull request and issue comments in 5,000 distinct Github accounts of which 527 have been identified as bots. Using this dataset we propose an automated classification model to detect bots, taking as main features the number of empty and non-empty comments of each account, the number of comment patterns, and the inequality between comments within comment patterns. We obtained a very high weighted average precision, recall and F1-score of 0.98 on a test set containing 40% of the data. We integrated the classification model into an open source command-line tool to allow practitioners to detect which accounts in a given Github repository actually correspond to bots. •Bots can be detected based on pull request and issue comments in GitHub repositories.•We proposed a ground-truth dataset of 5,000 GitHub accounts including 527 bots.•We developed a classification model that detects bots based on their comments.•We implemented an open source tool to detect bots in GitHub repositories.
AbstractList	Bots are frequently used in Github repositories to automate repetitive activities that are part of the distributed software development process. They communicate with human actors through comments. While detecting their presence is important for many reasons, no large and representative ground-truth dataset is available, nor are classification models to detect and validate bots on the basis of such a dataset. This paper proposes a ground-truth dataset, based on a manual analysis with high interrater agreement, of pull request and issue comments in 5,000 distinct Github accounts of which 527 have been identified as bots. Using this dataset we propose an automated classification model to detect bots, taking as main features the number of empty and non-empty comments of each account, the number of comment patterns, and the inequality between comments within comment patterns. We obtained a very high weighted average precision, recall and F1-score of 0.98 on a test set containing 40% of the data. We integrated the classification model into an open source command-line tool to allow practitioners to detect which accounts in a given Github repository actually correspond to bots. •Bots can be detected based on pull request and issue comments in GitHub repositories.•We proposed a ground-truth dataset of 5,000 GitHub accounts including 527 bots.•We developed a classification model that detects bots based on their comments.•We implemented an open source tool to detect bots in GitHub repositories.
ArticleNumber	110911
Author	Decan, Alexandre Mens, Tom Golzadeh, Mehdi Legay, Damien
Author_xml	– sequence: 1 givenname: Mehdi orcidid: 0000-0003-1041-439X surname: Golzadeh fullname: Golzadeh, Mehdi email: mehdi.golzadeh@umons.ac.be – sequence: 2 givenname: Alexandre orcidid: 0000-0002-5824-5823 surname: Decan fullname: Decan, Alexandre email: alexandre.decan@umons.ac.be – sequence: 3 givenname: Damien orcidid: 0000-0001-6811-6585 surname: Legay fullname: Legay, Damien email: damien.legay@umons.ac.be – sequence: 4 givenname: Tom orcidid: 0000-0003-3636-5020 surname: Mens fullname: Mens, Tom email: tom.mens@umons.ac.be
BookMark	eNp9kM9KAzEQh4NUsFYfwFteYGsm293u4qkUbYWCInoO-TNbs2wTSVLBtzdtPXnoaQbm9w0z3zUZOe-QkDtgU2BQ3_fTPsYpZxymAKwFuCBjaOZlAZw3IzLOmVnugV-R6xh7xtg8h8dELeg2-L0zRQr79EmNTDJiotIZqgcZo-2slsl6R3fe4EA7H6jBhDpZt6XKp0itoyub1ntFbYx7PLKvb1T73Q5dijfkspNDxNu_OiEfT4_vy3WxeVk9LxebQpd1mQqsq1a1ilVt24JqsKqglqrJV3JUFYDRM8Nlnne67RpVa67rEmrddKbjgHU5IXDaq4OPMWAnvoLdyfAjgImDJNGLLEkcJImTpMzM_zHapuO7KUg7nCUfTiTml74tBhG1RafR2JDlCOPtGfoXwTCEZA
CitedBy_id	crossref_primary_10_1016_j_infsof_2024_107425 crossref_primary_10_1177_26339137241231912 crossref_primary_10_3390_app122412594 crossref_primary_10_1109_TSE_2024_3353297 crossref_primary_10_1109_ACCESS_2022_3232982 crossref_primary_10_1016_j_jss_2024_112163 crossref_primary_10_1007_s10664_023_10299_7 crossref_primary_10_1145_3476042 crossref_primary_10_7717_peerj_cs_866 crossref_primary_10_1016_j_jss_2024_112287 crossref_primary_10_1145_3705303 crossref_primary_10_1109_MS_2022_3178601 crossref_primary_10_7717_peerj_cs_849 crossref_primary_10_1007_s10664_023_10336_5 crossref_primary_10_1145_3643773 crossref_primary_10_1109_MS_2022_3184430 crossref_primary_10_1109_MS_2022_3180297 crossref_primary_10_1111_tops_12613 crossref_primary_10_1007_s10664_022_10143_4 crossref_primary_10_1109_TSE_2024_3443741 crossref_primary_10_1007_s10664_024_10459_3 crossref_primary_10_1007_s10664_022_10205_7 crossref_primary_10_1145_3569949 crossref_primary_10_1007_s10664_023_10305_y crossref_primary_10_1109_TSE_2022_3156071 crossref_primary_10_1145_3729392 crossref_primary_10_1007_s10664_024_10608_8 crossref_primary_10_1109_TSE_2024_3410292 crossref_primary_10_1007_s10664_023_10369_w crossref_primary_10_1038_s41597_022_01819_z crossref_primary_10_1145_3593803 crossref_primary_10_1109_TSE_2024_3364675 crossref_primary_10_3390_app122110773
Cites_doi	10.1023/A:1010933404324 10.1145/3001896 10.1016/j.cose.2020.101715 10.1109/ICACCI.2016.7732476 10.1111/j.1469-8137.1912.tb05611.x 10.1017/S1351324916000243 10.1109/MS.2017.4541027 10.1016/j.im.2016.11.008 10.1214/aoms/1177730491 10.1145/3110025.3110163 10.2307/1924845 10.2307/2529310 10.1109/TKDE.2008.239 10.2307/2983054 10.1109/21.97458 10.1093/mind/LIX.236.433 10.1023/A:1022689900470 10.11613/BM.2012.031 10.1177/0049124113500475 10.1016/j.chb.2017.05.045 10.1037/0033-2909.114.3.494
ContentType	Journal Article
Copyright	2021 Elsevier Inc.
Copyright_xml	– notice: 2021 Elsevier Inc.
DBID	AAYXX CITATION
DOI	10.1016/j.jss.2021.110911
DatabaseName	CrossRef
DatabaseTitle	CrossRef
DatabaseTitleList
DeliveryMethod	fulltext_linktorsrc
Discipline	Computer Science
EISSN	1873-1228
ExternalDocumentID	10_1016_j_jss_2021_110911 S016412122100008X
GrantInformation_xml	– fundername: Fonds de la Recherche Scientifique – FNRS grantid: T.0017.18; O.0157.18F-RG43; J.0151.20 funderid: http://dx.doi.org/10.13039/501100002661
GroupedDBID	--K --M -~X .DC .~1 0R~ 1B1 1~. 1~5 29L 4.4 457 4G. 5GY 5VS 7-5 71M 8P~ 9JN 9M8 AABNK AACTN AAEDT AAEDW AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXUO AAYFN AAYOK ABBOA ABEFU ABFNM ABFRF ABFSI ABJNI ABMAC ABTAH ABXDB ABYKQ ACDAQ ACGFO ACGFS ACGOD ACNNM ACRLP ACZNC ADBBV ADEZE ADHUB ADJOM ADMUD AEBSH AEFWE AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHZHX AI. AIALX AIEXJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ASPBG AVWKF AXJTR AZFZN BKOJK BKOMP BLXMC CS3 DU5 E.L EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 FDB FEDTE FGOYB FIRID FNPLU FYGXN G-Q G8K GBLVA GBOLZ HLZ HVGLF HZ~ IHE J1W KOM LG9 M41 MO0 MS~ N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. PQQKQ Q38 R2- RIG RNS ROL RPZ RXW SBC SDF SDG SDP SES SEW SPC SPCBC SSV SSZ T5K TAE TN5 TWZ UHS UNMZH VH1 WUQ XPP ZMT ZY4 ~G- 9DU AATTM AAXKI AAYWO AAYXX ABDPE ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP CITATION EFKBS ~HD
ID	FETCH-LOGICAL-c363t-e659b9b059991b8e5516ab82022eb511dc4d2ab05fc9f8b6c2c6316c8fdf21e63
ISICitedReferencesCount	56
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000623099500003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0164-1212
IngestDate	Tue Nov 18 21:45:37 EST 2025 Sat Nov 29 07:24:04 EST 2025 Fri Feb 23 02:46:18 EST 2024
IsPeerReviewed	true
IsScholarly	true
Keywords	Bot identification Distributed software development Classification model Text similarity GitHub repositories
Language	English
LinkModel	OpenURL
MergedId	FETCHMERGED-LOGICAL-c363t-e659b9b059991b8e5516ab82022eb511dc4d2ab05fc9f8b6c2c6316c8fdf21e63
ORCID	0000-0003-3636-5020 0000-0003-1041-439X 0000-0002-5824-5823 0000-0001-6811-6585
ParticipantIDs	crossref_primary_10_1016_j_jss_2021_110911 crossref_citationtrail_10_1016_j_jss_2021_110911 elsevier_sciencedirect_doi_10_1016_j_jss_2021_110911
PublicationCentury	2000
PublicationDate	May 2021 2021-05-00
PublicationDateYYYYMMDD	2021-05-01
PublicationDate_xml	– month: 05 year: 2021 text: May 2021
PublicationDecade	2020
PublicationTitle	The Journal of systems and software
PublicationYear	2021
Publisher	Elsevier Inc
Publisher_xml	– name: Elsevier Inc
References	Romero, Parra (b63) 2020 Cosley, Frankowski, Terveen, Riedl (b13) 2007 Balachandran (b4) 2013 Gousios, Pinzger, Deursen (b33) 2014 He, Garcia (b37) 2009; 21 Jaccard (b40) 1912; 11 Hofmann, Kafadar, Wickham (b38) 2011 Dabbish, Stuart, Tsay, Herbsleb (b14) 2012 Liu, Gao, Chen, Nie, Liu (b53) 2019; 14 Frank, Hall (b27) 2001 Perez-Soler, Guerra, De Lara, Jurado (b59) 2017 Thomas, N.T., 2016. An e-business chatbot using AIML and LSA. In: International Conference on Advances in Computing, Communications and Informatics, ICACCI. pp. 2740–2742. Dey, Ma, Mockus (b16) 2019 Mann, Whitney (b54) 1947 Cliff (b12) 1993; 114 Lebeuf (b46) 2018 Gunn (b36) 1998 Minnich, A., Chavoshi, N., Koutra, D., Mueen, A., 2017. BotWalk: Efficient adaptive exploration of Twitter bot networks. In: IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM. pp. 467–474. Storey, Zagalsky (b65) 2016 Levenshtein (b50) 1966; 10 Breiman (b8) 2001; 45 Pedregosa, Varoquaux, Gramfort, Michel, Thirion, Grisel, Blondel, Prettenhofer, Weiss, Dubourg, Vanderplas, Passos, Cournapeau, Brucher, Perrot, Duchesnay (b58) 2011; 12 Urli, Yu, Seinturier, Monperrus (b70) 2018 Campbell, Quincy, Osserman, Pedersen (b11) 2013; 42 Grbac, Mausa, Basic (b35) 2013; vol. 1053 Tsay, Dabbish, Herbsleb (b68) 2014 Erlenhov, Neto, Chukaleski, Daknache (b23) 2020 Ben Mimoun, Poncin, Garnier (b5) 2017; 54 Golzadeh, Legay, Decan, Mens (b32) 2020 Hu, Gehringer (b39) 2019 McHugh (b55) 2012; 22 Ester, Kriegel, Sander, Xu (b25) 1996 Aha, Kibler, Albert (b2) 1991; 6 Burridge (b10) 1991; 154 Romano, J., Kromrey, J.D., Coraggio, J., Skowronek, J., Devine, L., 2006. Exploring methods for evaluating group differences on the NSSE and other surveys: Are the t-test and Cohen’s d indices the most appropriate choices? In: Annual Meeting of the Southern Association for Institutional Research. Wessel, De Souza, Steinmacher, Wiese, Polato, Chaves, Gerosa (b71) 2018 Gnewuch, Morana, Mädche (b30) 2017 Rebai, Ben Sghaier, Alizadeh, Kessentini, Chater (b60) 2019 Lin, Zagalsky, Storey, Serebrenik (b52) 2016 Wohlin, Runeson, Hst, Ohlsson, Regnell, Wessln (b74) 2012 Tsay, Dabbish, Herbsleb (b67) 2014 Katz (b43) 2020 Wessel, Steinmacher (b72) 2020 Gousios, Storey, Bacchelli (b34) 2016 Fryer, Ainley, Thompson, Gibson, Sherlock (b28) 2017; 75 Kalliamvakou, Gousios, Blincoe, Singer, German, Damian (b42) 2014 Lebeuf, Storey, Zagalsky (b49) 2018; 35 Safavian, Landgrebe (b64) 1991; 21 Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, Shyam, Sastry, Askell (b9) 2020 Lebeuf, Storey, Zagalsky (b47) 2017 Erlenhov, Gomes de Oliveira Neto, Scandariato, Leitner (b22) 2019 Dorfman (b19) 1979; 61 Benotti, Martínez, Schapachnik (b6) 2014 Amleshwaram, Reddy, Yadav, Gu, Yang (b3) 2013 Farooq, Grudin (b26) 2016; 23 Abokhodair, Yoo, McDonald (b1) 2015 Dale (b15) 2016; 22 Jain, Kota, Kumar, Patel (b41) 2018 Turing (b69) 1950; LIX Efthimion, Payne, Nicholas (b20) 2018; 1 Witten, Frank, Hall (b73) 2011 Rodríguez-Ruiz, Mata-Sánchez, Monroy, Loyola-González, López-Cuevas (b61) 2020; 91 Landis, Koch (b45) 1977; 33 Beschastnikh, Lungu, Zhuang (b7) 2017 Erlenhov, Neto, Leitner (b24) 2020 Dey, Mousavi, Ponce, Fry, Vasilescu, Filipova, Mockus (b17) 2020 Geiger (b29) 2013 Golzadeh, Decan, Mens (b31) 2019; vol. 2605 Mirhosseini, Parnin (b57) 2017 Kerry, Ellis, Bull (b44) 2009 Dominic, Houser, Steinmacher, Ritter, Rodeghero (b18) 2020 Lebeuf, Storey, Zagalsky (b48) 2017; 35 Elkan (b21) 2001 Lin, Ma, Huang (b51) 2020 Erlenhov (10.1016/j.jss.2021.110911_b23) 2020 Beschastnikh (10.1016/j.jss.2021.110911_b7) 2017 Dey (10.1016/j.jss.2021.110911_b17) 2020 Rodríguez-Ruiz (10.1016/j.jss.2021.110911_b61) 2020; 91 Cosley (10.1016/j.jss.2021.110911_b13) 2007 Lebeuf (10.1016/j.jss.2021.110911_b48) 2017; 35 He (10.1016/j.jss.2021.110911_b37) 2009; 21 Lebeuf (10.1016/j.jss.2021.110911_b46) 2018 Mann (10.1016/j.jss.2021.110911_b54) 1947 Dominic (10.1016/j.jss.2021.110911_b18) 2020 Campbell (10.1016/j.jss.2021.110911_b11) 2013; 42 Dey (10.1016/j.jss.2021.110911_b16) 2019 Burridge (10.1016/j.jss.2021.110911_b10) 1991; 154 Fryer (10.1016/j.jss.2021.110911_b28) 2017; 75 Tsay (10.1016/j.jss.2021.110911_b67) 2014 Wohlin (10.1016/j.jss.2021.110911_b74) 2012 Farooq (10.1016/j.jss.2021.110911_b26) 2016; 23 Gousios (10.1016/j.jss.2021.110911_b34) 2016 Wessel (10.1016/j.jss.2021.110911_b72) 2020 Golzadeh (10.1016/j.jss.2021.110911_b31) 2019; vol. 2605 Dabbish (10.1016/j.jss.2021.110911_b14) 2012 Storey (10.1016/j.jss.2021.110911_b65) 2016 Benotti (10.1016/j.jss.2021.110911_b6) 2014 Breiman (10.1016/j.jss.2021.110911_b8) 2001; 45 Hu (10.1016/j.jss.2021.110911_b39) 2019 Erlenhov (10.1016/j.jss.2021.110911_b22) 2019 Romero (10.1016/j.jss.2021.110911_b63) 2020 Katz (10.1016/j.jss.2021.110911_b43) 2020 Geiger (10.1016/j.jss.2021.110911_b29) 2013 Balachandran (10.1016/j.jss.2021.110911_b4) 2013 Lebeuf (10.1016/j.jss.2021.110911_b49) 2018; 35 Landis (10.1016/j.jss.2021.110911_b45) 1977; 33 Efthimion (10.1016/j.jss.2021.110911_b20) 2018; 1 Elkan (10.1016/j.jss.2021.110911_b21) 2001 Witten (10.1016/j.jss.2021.110911_b73) 2011 Mirhosseini (10.1016/j.jss.2021.110911_b57) 2017 Ester (10.1016/j.jss.2021.110911_b25) 1996 Perez-Soler (10.1016/j.jss.2021.110911_b59) 2017 Aha (10.1016/j.jss.2021.110911_b2) 1991; 6 Cliff (10.1016/j.jss.2021.110911_b12) 1993; 114 Ben Mimoun (10.1016/j.jss.2021.110911_b5) 2017; 54 Pedregosa (10.1016/j.jss.2021.110911_b58) 2011; 12 Gunn (10.1016/j.jss.2021.110911_b36) 1998 Dorfman (10.1016/j.jss.2021.110911_b19) 1979; 61 Urli (10.1016/j.jss.2021.110911_b70) 2018 Gnewuch (10.1016/j.jss.2021.110911_b30) 2017 10.1016/j.jss.2021.110911_b56 Abokhodair (10.1016/j.jss.2021.110911_b1) 2015 Amleshwaram (10.1016/j.jss.2021.110911_b3) 2013 Lin (10.1016/j.jss.2021.110911_b52) 2016 Hofmann (10.1016/j.jss.2021.110911_b38) 2011 Levenshtein (10.1016/j.jss.2021.110911_b50) 1966; 10 Kerry (10.1016/j.jss.2021.110911_b44) 2009 Liu (10.1016/j.jss.2021.110911_b53) 2019; 14 Erlenhov (10.1016/j.jss.2021.110911_b24) 2020 Golzadeh (10.1016/j.jss.2021.110911_b32) 2020 Jaccard (10.1016/j.jss.2021.110911_b40) 1912; 11 Frank (10.1016/j.jss.2021.110911_b27) 2001 Brown (10.1016/j.jss.2021.110911_b9) 2020 Grbac (10.1016/j.jss.2021.110911_b35) 2013; vol. 1053 Lebeuf (10.1016/j.jss.2021.110911_b47) 2017 Kalliamvakou (10.1016/j.jss.2021.110911_b42) 2014 Wessel (10.1016/j.jss.2021.110911_b71) 2018 Rebai (10.1016/j.jss.2021.110911_b60) 2019 10.1016/j.jss.2021.110911_b66 Gousios (10.1016/j.jss.2021.110911_b33) 2014 Lin (10.1016/j.jss.2021.110911_b51) 2020 Jain (10.1016/j.jss.2021.110911_b41) 2018 Dale (10.1016/j.jss.2021.110911_b15) 2016; 22 Safavian (10.1016/j.jss.2021.110911_b64) 1991; 21 McHugh (10.1016/j.jss.2021.110911_b55) 2012; 22 10.1016/j.jss.2021.110911_b62 Turing (10.1016/j.jss.2021.110911_b69) 1950; LIX Tsay (10.1016/j.jss.2021.110911_b68) 2014
References_xml	– volume: 42 start-page: 294 year: 2013 end-page: 320 ident: b11 article-title: Coding in-depth semistructured interviews: Problems of unitization and intercoder reliability and agreement publication-title: Sociol. Methods Res. – start-page: 66 year: 2020 end-page: 70 ident: b63 article-title: Experiences building an answer bot for Gitter publication-title: International Workshop on Bots in Software Engineering – start-page: 1 year: 2018 end-page: 6 ident: b41 article-title: Convey: Exploring the use of a context view for chatbots publication-title: Conference on Human Factors in Computing Systems – volume: 21 start-page: 1263 year: 2009 end-page: 1284 ident: b37 article-title: Learning from imbalanced data publication-title: IEEE Trans. Knowl. Data Eng. – year: 2012 ident: b74 article-title: Experimentation in Software Engineering – start-page: 92 year: 2014 end-page: 101 ident: b42 article-title: The promises and perils of mining GitHub publication-title: International Conference on Mining Software Repositories – volume: 33 start-page: 159 year: 1977 end-page: 174 ident: b45 article-title: The measurement of observer agreement for categorical data publication-title: Biometrics – start-page: 50 year: 1947 end-page: 60 ident: b54 article-title: On a test of whether one of two random variables is stochastically larger than the other publication-title: Ann. Math. Stat. – volume: 22 start-page: 811 year: 2016 end-page: 817 ident: b15 article-title: The return of the chatbots publication-title: Nat. Lang. Eng. – year: 2020 ident: b18 article-title: Conversational bot for newcomers onboarding to open source projects – start-page: 35 year: 2017 end-page: 38 ident: b7 article-title: Accelerating software engineering research adoption with analysis bots publication-title: International Conference on Software Engineering: New Ideas and Emerging Results Track – start-page: 51 year: 2013 end-page: 56 ident: b29 article-title: Are computers merely “supporting” cooperative work: Towards an ethnography of bot development publication-title: International Conference on Computer Supported Cooperative Work – start-page: 31 year: 2020 end-page: 35 ident: b32 article-title: Bot or not? Detecting bots in GitHub pull request activity based on comment similarity publication-title: International Workshop on Bots in Software Engineering – start-page: 36 year: 2020 end-page: 40 ident: b51 article-title: MSABot : A chatbot framework for assisting in the development and operation of microservice-based systems publication-title: International Conference on Software Engineering Workshops – reference: Thomas, N.T., 2016. An e-business chatbot using AIML and LSA. In: International Conference on Advances in Computing, Communications and Informatics, ICACCI. pp. 2740–2742. – volume: 91 year: 2020 ident: b61 article-title: A one-class classification approach for bot detection on Twitter publication-title: Comput. Secur. – volume: 35 start-page: 18 year: 2018 end-page: 23 ident: b49 article-title: Software bots publication-title: IEEE Softw. – year: 2020 ident: b9 article-title: Language models are few-shot learners – year: 2018 ident: b71 article-title: The power of bots: Understanding bots in OSS projects publication-title: The ACM International Conference on Human-Computer Interaction – year: 2020 ident: b23 article-title: Challenges and guidelines on designing test cases for test bots publication-title: International Workshop on Bots in Software Engineering – start-page: 723 year: 2017 end-page: 728 ident: b59 article-title: The rise of the (modelling) bots: Towards assisted modelling via social networks publication-title: IEEE/ACM International Conference on Automated Software Engineering – volume: 11 start-page: 37 year: 1912 end-page: 50 ident: b40 article-title: The distribution of the flora in the alpine zone publication-title: New Phytol. – start-page: 356 year: 2014 end-page: 366 ident: b67 article-title: Influence of social and technical factors for evaluating contribution in GitHub publication-title: International Conference on Software Engineering – start-page: 333 year: 2016 end-page: 336 ident: b52 article-title: Why developers are slacking off: Understanding how software teams use slack publication-title: ACM Conference on Computer Supported Cooperative Work – volume: 22 start-page: 276 year: 2012 end-page: 282 ident: b55 article-title: Interrater reliability: the kappa statistic publication-title: Biochem. Med. – volume: LIX start-page: 433 year: 1950 end-page: 460 ident: b69 article-title: Computing machinery and intelligence publication-title: Mind – volume: 54 start-page: 545 year: 2017 end-page: 559 ident: b5 article-title: Animated conversational agents and e-consumer productivity: The roles of agents and individual characteristics publication-title: Inf. Manag. – volume: 75 start-page: 461 year: 2017 end-page: 468 ident: b28 article-title: Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners publication-title: Comput. Hum. Behav. – year: 2011 ident: b73 article-title: Data Mining: Practical Machine Learning Tools and Techniques – year: 2020 ident: b17 article-title: Detecting and characterizing bots that commit code publication-title: International Conference on Mining Software Repositories – start-page: 36 year: 2019 end-page: 45 ident: b16 article-title: Patterns of effort contribution and demand and user classification based on participation patterns in NPM ecosystem publication-title: International Conference on Predictive Models and Data Analytics in Software Engineering – volume: 23 start-page: 26 year: 2016 end-page: 32 ident: b26 article-title: Human-computer integration publication-title: Interactions – start-page: 931 year: 2013 end-page: 940 ident: b4 article-title: Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation publication-title: International Conference on Software Engineering – year: 2020 ident: b72 article-title: The inconvenient side of software bots on pull requests publication-title: International Workshop on Bots in Software Engineering – volume: 12 start-page: 2825 year: 2011 end-page: 2830 ident: b58 article-title: Scikit-learn: Machine learning in Python publication-title: J. Mach. Learn. Res. – volume: 35 start-page: 18 year: 2017 end-page: 23 ident: b48 article-title: Software bots publication-title: IEEE Softw. – volume: 45 start-page: 5 year: 2001 end-page: 32 ident: b8 article-title: Random forests publication-title: Mach. Learn. – volume: 10 start-page: 707 year: 1966 end-page: 710 ident: b50 article-title: Binary codes capable of correcting deletions insertions and reversals publication-title: Sov. Phys. Dokl. – volume: 1 year: 2018 ident: b20 article-title: Supervised machine learning bot detection techniques to identify social Twitter bots publication-title: SMU Data Sci. Rev. – year: 2013 ident: b3 article-title: CATS: Characterizing automation of Twitter spammers publication-title: International Conference on Communication Systems and Networks – volume: vol. 2605 year: 2019 ident: b31 article-title: On the effect of discussions on pull request decisions publication-title: The 18th Belgium-Netherlands Software Evolution Workshop – start-page: 145 year: 2001 end-page: 156 ident: b27 article-title: A simple approach to ordinal classification publication-title: European Confernece on Machine Learning – start-page: 169 year: 2009 end-page: 182 ident: b44 article-title: Conversational agents in E-learning publication-title: Applications and Innovations in Intelligent Systems, Vol. XVI – start-page: 1277 year: 2012 end-page: 1286 ident: b14 article-title: Social coding in GitHub: Transparency and collaboration in an open software repository publication-title: ACM Conference on Computer Supported Cooperative Work – year: 1998 ident: b36 article-title: Support Vector Machines for Classification and Regression – volume: 21 start-page: 660 year: 1991 end-page: 674 ident: b64 article-title: A survey of decision tree classifier methodology publication-title: IEEE Trans. Syst. Man Cybern. – start-page: 7 year: 2019 end-page: 11 ident: b22 article-title: Current and future bots in software development publication-title: International Workshop on Bots in Software Engineering – volume: 14 start-page: 1 year: 2019 end-page: 14 ident: b53 article-title: ATOM: Commit message generation based on abstract syntax tree and hybrid ranking publication-title: IIEEE Softw. – start-page: 445 year: 2020 end-page: 455 ident: b24 article-title: An empirical study of bots in software development: Characteristics and challenges from a practitioner’s perspective publication-title: ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering – year: 2017 ident: b30 article-title: Towards designing cooperative and social conversational agents for customer service publication-title: International Conference on Information Systems – start-page: 63 year: 2014 end-page: 68 ident: b6 article-title: Engaging high school students using chatbots publication-title: Conference on Innovation and Technology in Computer Science Education – start-page: 95 year: 2018 end-page: 104 ident: b70 article-title: How to design a program repair bot?: Insights from the repairnator project publication-title: International Conference on Software Engineering – start-page: 226 year: 1996 end-page: -231 ident: b25 article-title: A density-based algorithm for discovering clusters in large spatial databases with noise publication-title: International Conference on Knowledge Discovery and Data Mining – start-page: 928 year: 2016 end-page: 931 ident: b65 article-title: Disrupting developer productivity one Bot at a Time publication-title: ACM SIGSOFT Symposium on the Foundations of Software Engineering – start-page: 345 year: 2014 end-page: 355 ident: b33 article-title: An exploratory study of the pull-based software development model publication-title: International Conference on Software Engineering – start-page: 1 year: 2019 end-page: 9 ident: b39 article-title: Improving feedback on GitHub pull requests: A bots approach publication-title: Frontiers in Education Conference – volume: 61 start-page: 146 year: 1979 end-page: 149 ident: b19 article-title: A formula for the Gini coefficient publication-title: Rev. Econ. Stat. – start-page: 973 year: 2001 end-page: 978 ident: b21 article-title: The foundations of cost-sensitive learning publication-title: International Joint Conference on Artificial Intelligence – reference: Minnich, A., Chavoshi, N., Koutra, D., Mueen, A., 2017. BotWalk: Efficient adaptive exploration of Twitter bot networks. In: IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM. pp. 467–474. – start-page: 84 year: 2017 end-page: 94 ident: b57 article-title: Can automated pull requests encourage software developers to upgrade out-of-date dependencies? publication-title: International Conference on Automated Software Engineering – year: 2020 ident: b43 article-title: Libraries.io open source repository and dependency metadata (version 1.6.0) – start-page: 32 year: 2007 end-page: 41 ident: b13 article-title: SuggestBot: Using intelligent task routing to help people find work in Wikipedia publication-title: International Conference on Intelligent User Interfaces – start-page: 285 year: 2016 end-page: 296 ident: b34 article-title: Work practices and challenges in pull-based development: The contributor’s perspective publication-title: International Conference on Software Engineering – start-page: 152 year: 2019 end-page: 162 ident: b60 article-title: Interactive refactoring documentation bot publication-title: International Working Conference on Source Code Analysis and Manipulation – volume: 114 start-page: 494 year: 1993 end-page: 509 ident: b12 article-title: Dominance statistics: Ordinal analyses to answer ordinal questions publication-title: Psychol. Bull. – volume: 6 start-page: 37 year: 1991 end-page: 66 ident: b2 article-title: Instance-based learning algorithms publication-title: Mach. Learn. – year: 2017 ident: b47 article-title: How software developers mitigate collaboration friction with chatbots – year: 2011 ident: b38 article-title: Letter-Value Plots: Boxplots for Large Data – volume: vol. 1053 year: 2013 ident: b35 article-title: Stability of software defect prediction in relation to levels of data imbalance publication-title: Workshop on Software Quality Analysis, Monitoring, Improvement, and Applications – reference: Romano, J., Kromrey, J.D., Coraggio, J., Skowronek, J., Devine, L., 2006. Exploring methods for evaluating group differences on the NSSE and other surveys: Are the t-test and Cohen’s d indices the most appropriate choices? In: Annual Meeting of the Southern Association for Institutional Research. – start-page: 839 year: 2015 end-page: 851 ident: b1 article-title: Dissecting a social Botnet: Growth, content and influence in Twitter publication-title: ACM Conference on Computer Supported Cooperative Work and Social Computing – year: 2018 ident: b46 article-title: A Taxonomy of Software Bots: Towards a Deeper Understanding of Software Bot Characteristics – start-page: 144 year: 2014 end-page: 154 ident: b68 article-title: Let’s talk about it: Evaluating contributions through discussion in GitHub publication-title: ACM SIGSOFT Symposium on Foundations of Software Engineering – volume: 154 start-page: 361 year: 1991 end-page: 364 ident: b10 publication-title: J. R. Stat. Soc. Ser. A (Stat. Soc.) – year: 2017 ident: 10.1016/j.jss.2021.110911_b30 article-title: Towards designing cooperative and social conversational agents for customer service – year: 1998 ident: 10.1016/j.jss.2021.110911_b36 – start-page: 928 year: 2016 ident: 10.1016/j.jss.2021.110911_b65 article-title: Disrupting developer productivity one Bot at a Time – volume: 12 start-page: 2825 year: 2011 ident: 10.1016/j.jss.2021.110911_b58 article-title: Scikit-learn: Machine learning in Python publication-title: J. Mach. Learn. Res. – start-page: 345 year: 2014 ident: 10.1016/j.jss.2021.110911_b33 article-title: An exploratory study of the pull-based software development model – start-page: 63 year: 2014 ident: 10.1016/j.jss.2021.110911_b6 article-title: Engaging high school students using chatbots – start-page: 169 year: 2009 ident: 10.1016/j.jss.2021.110911_b44 article-title: Conversational agents in E-learning – volume: 45 start-page: 5 issue: 1 year: 2001 ident: 10.1016/j.jss.2021.110911_b8 article-title: Random forests publication-title: Mach. Learn. doi: 10.1023/A:1010933404324 – year: 2020 ident: 10.1016/j.jss.2021.110911_b43 – volume: 23 start-page: 26 issue: 6 year: 2016 ident: 10.1016/j.jss.2021.110911_b26 article-title: Human-computer integration publication-title: Interactions doi: 10.1145/3001896 – volume: 1 issue: 2 year: 2018 ident: 10.1016/j.jss.2021.110911_b20 article-title: Supervised machine learning bot detection techniques to identify social Twitter bots publication-title: SMU Data Sci. Rev. – volume: 91 year: 2020 ident: 10.1016/j.jss.2021.110911_b61 article-title: A one-class classification approach for bot detection on Twitter publication-title: Comput. Secur. doi: 10.1016/j.cose.2020.101715 – start-page: 445 year: 2020 ident: 10.1016/j.jss.2021.110911_b24 article-title: An empirical study of bots in software development: Characteristics and challenges from a practitioner’s perspective – ident: 10.1016/j.jss.2021.110911_b66 doi: 10.1109/ICACCI.2016.7732476 – start-page: 144 year: 2014 ident: 10.1016/j.jss.2021.110911_b68 article-title: Let’s talk about it: Evaluating contributions through discussion in GitHub – volume: 10 start-page: 707 issue: 8 year: 1966 ident: 10.1016/j.jss.2021.110911_b50 article-title: Binary codes capable of correcting deletions insertions and reversals publication-title: Sov. Phys. Dokl. – volume: 11 start-page: 37 issue: 2 year: 1912 ident: 10.1016/j.jss.2021.110911_b40 article-title: The distribution of the flora in the alpine zone publication-title: New Phytol. doi: 10.1111/j.1469-8137.1912.tb05611.x – volume: 22 start-page: 811 issue: 5 year: 2016 ident: 10.1016/j.jss.2021.110911_b15 article-title: The return of the chatbots publication-title: Nat. Lang. Eng. doi: 10.1017/S1351324916000243 – year: 2012 ident: 10.1016/j.jss.2021.110911_b74 – start-page: 32 year: 2007 ident: 10.1016/j.jss.2021.110911_b13 article-title: SuggestBot: Using intelligent task routing to help people find work in Wikipedia – year: 2018 ident: 10.1016/j.jss.2021.110911_b71 article-title: The power of bots: Understanding bots in OSS projects – start-page: 152 year: 2019 ident: 10.1016/j.jss.2021.110911_b60 article-title: Interactive refactoring documentation bot – start-page: 7 year: 2019 ident: 10.1016/j.jss.2021.110911_b22 article-title: Current and future bots in software development – start-page: 35 year: 2017 ident: 10.1016/j.jss.2021.110911_b7 article-title: Accelerating software engineering research adoption with analysis bots – start-page: 973 year: 2001 ident: 10.1016/j.jss.2021.110911_b21 article-title: The foundations of cost-sensitive learning – volume: vol. 1053 year: 2013 ident: 10.1016/j.jss.2021.110911_b35 article-title: Stability of software defect prediction in relation to levels of data imbalance – volume: 35 start-page: 18 issue: 1 year: 2017 ident: 10.1016/j.jss.2021.110911_b48 article-title: Software bots publication-title: IEEE Softw. doi: 10.1109/MS.2017.4541027 – volume: 54 start-page: 545 issue: 5 year: 2017 ident: 10.1016/j.jss.2021.110911_b5 article-title: Animated conversational agents and e-consumer productivity: The roles of agents and individual characteristics publication-title: Inf. Manag. doi: 10.1016/j.im.2016.11.008 – year: 2017 ident: 10.1016/j.jss.2021.110911_b47 – start-page: 50 year: 1947 ident: 10.1016/j.jss.2021.110911_b54 article-title: On a test of whether one of two random variables is stochastically larger than the other publication-title: Ann. Math. Stat. doi: 10.1214/aoms/1177730491 – start-page: 356 year: 2014 ident: 10.1016/j.jss.2021.110911_b67 article-title: Influence of social and technical factors for evaluating contribution in GitHub – start-page: 95 year: 2018 ident: 10.1016/j.jss.2021.110911_b70 article-title: How to design a program repair bot?: Insights from the repairnator project – start-page: 1277 year: 2012 ident: 10.1016/j.jss.2021.110911_b14 article-title: Social coding in GitHub: Transparency and collaboration in an open software repository – start-page: 333 year: 2016 ident: 10.1016/j.jss.2021.110911_b52 article-title: Why developers are slacking off: Understanding how software teams use slack – ident: 10.1016/j.jss.2021.110911_b56 doi: 10.1145/3110025.3110163 – year: 2018 ident: 10.1016/j.jss.2021.110911_b46 – start-page: 36 year: 2019 ident: 10.1016/j.jss.2021.110911_b16 article-title: Patterns of effort contribution and demand and user classification based on participation patterns in NPM ecosystem – volume: 61 start-page: 146 issue: 1 year: 1979 ident: 10.1016/j.jss.2021.110911_b19 article-title: A formula for the Gini coefficient publication-title: Rev. Econ. Stat. doi: 10.2307/1924845 – start-page: 931 year: 2013 ident: 10.1016/j.jss.2021.110911_b4 article-title: Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation – start-page: 1 year: 2018 ident: 10.1016/j.jss.2021.110911_b41 article-title: Convey: Exploring the use of a context view for chatbots – year: 2020 ident: 10.1016/j.jss.2021.110911_b23 article-title: Challenges and guidelines on designing test cases for test bots – start-page: 51 year: 2013 ident: 10.1016/j.jss.2021.110911_b29 article-title: Are computers merely “supporting” cooperative work: Towards an ethnography of bot development – volume: 14 start-page: 1 issue: 8 year: 2019 ident: 10.1016/j.jss.2021.110911_b53 article-title: ATOM: Commit message generation based on abstract syntax tree and hybrid ranking publication-title: IIEEE Softw. – volume: 33 start-page: 159 issue: 1 year: 1977 ident: 10.1016/j.jss.2021.110911_b45 article-title: The measurement of observer agreement for categorical data publication-title: Biometrics doi: 10.2307/2529310 – start-page: 31 year: 2020 ident: 10.1016/j.jss.2021.110911_b32 article-title: Bot or not? Detecting bots in GitHub pull request activity based on comment similarity – volume: 21 start-page: 1263 issue: 9 year: 2009 ident: 10.1016/j.jss.2021.110911_b37 article-title: Learning from imbalanced data publication-title: IEEE Trans. Knowl. Data Eng. doi: 10.1109/TKDE.2008.239 – year: 2020 ident: 10.1016/j.jss.2021.110911_b72 article-title: The inconvenient side of software bots on pull requests – volume: 154 start-page: 361 issue: 2 year: 1991 ident: 10.1016/j.jss.2021.110911_b10 publication-title: J. R. Stat. Soc. Ser. A (Stat. Soc.) doi: 10.2307/2983054 – start-page: 1 year: 2019 ident: 10.1016/j.jss.2021.110911_b39 article-title: Improving feedback on GitHub pull requests: A bots approach – start-page: 84 year: 2017 ident: 10.1016/j.jss.2021.110911_b57 article-title: Can automated pull requests encourage software developers to upgrade out-of-date dependencies? – year: 2013 ident: 10.1016/j.jss.2021.110911_b3 article-title: CATS: Characterizing automation of Twitter spammers – start-page: 723 year: 2017 ident: 10.1016/j.jss.2021.110911_b59 article-title: The rise of the (modelling) bots: Towards assisted modelling via social networks – volume: 21 start-page: 660 year: 1991 ident: 10.1016/j.jss.2021.110911_b64 article-title: A survey of decision tree classifier methodology publication-title: IEEE Trans. Syst. Man Cybern. doi: 10.1109/21.97458 – volume: 35 start-page: 18 issue: 1 year: 2018 ident: 10.1016/j.jss.2021.110911_b49 article-title: Software bots publication-title: IEEE Softw. doi: 10.1109/MS.2017.4541027 – start-page: 92 year: 2014 ident: 10.1016/j.jss.2021.110911_b42 article-title: The promises and perils of mining GitHub – volume: LIX start-page: 433 issue: 236 year: 1950 ident: 10.1016/j.jss.2021.110911_b69 article-title: Computing machinery and intelligence publication-title: Mind doi: 10.1093/mind/LIX.236.433 – volume: 6 start-page: 37 issue: 1 year: 1991 ident: 10.1016/j.jss.2021.110911_b2 article-title: Instance-based learning algorithms publication-title: Mach. Learn. doi: 10.1023/A:1022689900470 – year: 2011 ident: 10.1016/j.jss.2021.110911_b73 – volume: 22 start-page: 276 issue: 3 year: 2012 ident: 10.1016/j.jss.2021.110911_b55 article-title: Interrater reliability: the kappa statistic publication-title: Biochem. Med. doi: 10.11613/BM.2012.031 – start-page: 145 year: 2001 ident: 10.1016/j.jss.2021.110911_b27 article-title: A simple approach to ordinal classification – start-page: 285 year: 2016 ident: 10.1016/j.jss.2021.110911_b34 article-title: Work practices and challenges in pull-based development: The contributor’s perspective – volume: 42 start-page: 294 issue: 3 year: 2013 ident: 10.1016/j.jss.2021.110911_b11 article-title: Coding in-depth semistructured interviews: Problems of unitization and intercoder reliability and agreement publication-title: Sociol. Methods Res. doi: 10.1177/0049124113500475 – year: 2020 ident: 10.1016/j.jss.2021.110911_b17 article-title: Detecting and characterizing bots that commit code – start-page: 36 year: 2020 ident: 10.1016/j.jss.2021.110911_b51 article-title: MSABot : A chatbot framework for assisting in the development and operation of microservice-based systems – year: 2020 ident: 10.1016/j.jss.2021.110911_b9 – volume: 75 start-page: 461 year: 2017 ident: 10.1016/j.jss.2021.110911_b28 article-title: Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners publication-title: Comput. Hum. Behav. doi: 10.1016/j.chb.2017.05.045 – volume: 114 start-page: 494 issue: 3 year: 1993 ident: 10.1016/j.jss.2021.110911_b12 article-title: Dominance statistics: Ordinal analyses to answer ordinal questions publication-title: Psychol. Bull. doi: 10.1037/0033-2909.114.3.494 – year: 2011 ident: 10.1016/j.jss.2021.110911_b38 – start-page: 226 year: 1996 ident: 10.1016/j.jss.2021.110911_b25 article-title: A density-based algorithm for discovering clusters in large spatial databases with noise – volume: vol. 2605 year: 2019 ident: 10.1016/j.jss.2021.110911_b31 article-title: On the effect of discussions on pull request decisions – ident: 10.1016/j.jss.2021.110911_b62 – year: 2020 ident: 10.1016/j.jss.2021.110911_b18 – start-page: 839 year: 2015 ident: 10.1016/j.jss.2021.110911_b1 article-title: Dissecting a social Botnet: Growth, content and influence in Twitter – start-page: 66 year: 2020 ident: 10.1016/j.jss.2021.110911_b63 article-title: Experiences building an answer bot for Gitter
SSID	ssj0007202
Score	2.576109
Snippet	Bots are frequently used in Github repositories to automate repetitive activities that are part of the distributed software development process. They...
SourceID	crossref elsevier
SourceType	Enrichment Source Index Database Publisher
StartPage	110911
SubjectTerms	Bot identification Classification model Distributed software development GitHub repositories Text similarity
Title	A ground-truth dataset and classification model for detecting bots in GitHub issue and PR comments
URI	https://dx.doi.org/10.1016/j.jss.2021.110911
Volume	175
WOSCitedRecordID	wos000623099500003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVESC databaseName: ScienceDirect database customDbUrl: eissn: 1873-1228 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0007202 issn: 0164-1212 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3JbtswECXcpIdeuhdJN_DQUw0FJrWRR6NIkxZNEAQu4JvATY0MRw5iZUG-oJ_dISlKdtoUzaEXwZBFWtB7GD0P3wwR-sBUzjkVJJKSZPAHRY8ilgkWGSZTk2ieykS6zSbyw0M2nfKjweBnqIW5nOd1za6v-dl_hRrOAdi2dPYecHeTwgn4DKDDEWCH4z8BPx7aSo1aR835RXMytBbQpfE-cmWlsvUGedTdLjjOZ6iNXUuwWQO5aJxDdq9q9i_k0MHiKwqOrf38NLR-CoK2Ly1zotY3hvZ9n5cQ4a9Eb67dW8xvhDYuj3NgTnTViWijfBq2rbbph3wzP4Rf5henVV-0dmC8-p_4xxeyFpT0HsGQyMxsv0SyHonzdCWW2l6oPhD_FuZ9xmG2M1vajuuU7PTXrrfUvvWq6wyIwds2K2CKwk5R-CkeoE2apxzi4-b4y-70a_dWz6nzr3b3HVbInVfw1n38WeOs6JbJU_S4xQaPPVGeoYGpn6MnYTMP3Mb2F0iO8SpvcMsbDHjgdd5gxxsMvMEdb7DlDa5q7HmDHW_c2KNjHHjzEn3_vDv5tB-1G3BEKs7iJjJZyiWXtoUPJ5IZu6gqJGhGSo0Epa5VoqmA70vFSyYzRVUWk0yxUpeUmCx-hTbqRW22EE4Vg1cDYUySMmEggso81cKVQscgeUfbaBQeWaHa7vR2k5R5cSdU2-hjN-TMt2b528VJwKFotaXXjAVw6u5hr-_zG2_Qo57qb9EGwGXeoYfqsqmW5-9bQv0CDqeZog
linkProvider	Elsevier
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+ground-truth+dataset+and+classification+model+for+detecting+bots+in+GitHub+issue+and+PR+comments&rft.jtitle=The+Journal+of+systems+and+software&rft.au=Golzadeh%2C+Mehdi&rft.au=Decan%2C+Alexandre&rft.au=Legay%2C+Damien&rft.au=Mens%2C+Tom&rft.date=2021-05-01&rft.issn=0164-1212&rft.volume=175&rft.spage=110911&rft_id=info:doi/10.1016%2Fj.jss.2021.110911&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_jss_2021_110911
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0164-1212&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0164-1212&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0164-1212&client=summon