Can AI serve as a substitute for human subjects in software engineering research?
Research within sociotechnical domains, such as software engineering, fundamentally requires the human perspective. Nevertheless, traditional qualitative data collection methods suffer from difficulties in participant recruitment, scaling, and labor intensity. This vision paper proposes a novel appr...
Uložené v:
| Vydané v: | Automated software engineering Ročník 31; číslo 1; s. 13 |
|---|---|
| Hlavní autori: | , , , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
New York
Springer US
01.05.2024
Springer Nature B.V |
| Predmet: | |
| ISSN: | 0928-8910, 1573-7535 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Research within sociotechnical domains, such as software engineering, fundamentally requires the human perspective. Nevertheless, traditional qualitative data collection methods suffer from difficulties in participant recruitment, scaling, and labor intensity. This vision paper proposes a novel approach to qualitative data collection in software engineering research by harnessing the capabilities of artificial intelligence (AI), especially large language models (LLMs) like ChatGPT and multimodal foundation models. We explore the potential of AI-generated synthetic text as an alternative source of qualitative data, discussing how LLMs can replicate human responses and behaviors in research settings. We discuss AI applications in emulating humans in interviews, focus groups, surveys, observational studies, and user evaluations. We discuss open problems and research opportunities to implement this vision. In the future, an integrated approach where both AI and human-generated data coexist will likely yield the most effective outcomes. |
|---|---|
| AbstractList | Research within sociotechnical domains, such as software engineering, fundamentally requires the human perspective. Nevertheless, traditional qualitative data collection methods suffer from difficulties in participant recruitment, scaling, and labor intensity. This vision paper proposes a novel approach to qualitative data collection in software engineering research by harnessing the capabilities of artificial intelligence (AI), especially large language models (LLMs) like ChatGPT and multimodal foundation models. We explore the potential of AI-generated synthetic text as an alternative source of qualitative data, discussing how LLMs can replicate human responses and behaviors in research settings. We discuss AI applications in emulating humans in interviews, focus groups, surveys, observational studies, and user evaluations. We discuss open problems and research opportunities to implement this vision. In the future, an integrated approach where both AI and human-generated data coexist will likely yield the most effective outcomes. |
| ArticleNumber | 13 |
| Author | Gerosa, Marco Trinkenreich, Bianca Steinmacher, Igor Sarma, Anita |
| Author_xml | – sequence: 1 givenname: Marco surname: Gerosa fullname: Gerosa, Marco email: Marco.Gerosa@nau.edu organization: Northern Arizona University – sequence: 2 givenname: Bianca surname: Trinkenreich fullname: Trinkenreich, Bianca organization: Oregon State University – sequence: 3 givenname: Igor surname: Steinmacher fullname: Steinmacher, Igor organization: Northern Arizona University – sequence: 4 givenname: Anita surname: Sarma fullname: Sarma, Anita organization: Oregon State University |
| BookMark | eNp9kE1LAzEQhoNUsK3-AU8Bz6uTzSa7OUkpfhQEEfQcsumkTWl3a5JV_PduXUHw0NMMw_vMDM-EjJq2QUIuGVwzgPImMhBMZJDzDKAAlckTMmai5FkpuBiRMai8yirF4IxMYtwAgJJKjcnL3DR0tqARwwdSE6mhsatj8qlLSF0b6Lrb9ZF-uEGbIvV937r0aQJSbFa-QQy-WdGAEU2w69tzcurMNuLFb52St_u71_lj9vT8sJjPnjLLmUqZc0q63BlboxHMMljWQqATEnJwsjLC1Dx3kpW5EW5ZAVqHFVrJal4WgAWfkqth7z607x3GpDdtF5r-pOZQVLmQXMk-lQ8pG9oYAzq9D35nwpdmoA_q9KBO9-r0jzp9gKp_kPXJJN82KRi_PY7yAY37gxYMf18dob4BmXOFlg |
| CitedBy_id | crossref_primary_10_2139_ssrn_4595896 crossref_primary_10_17645_mac_9677 crossref_primary_10_1145_3712301 crossref_primary_10_34133_hds_0284 |
| Cites_doi | 10.1126/science.adj6791 10.18653/v1/2023.findings-emnlp.669 10.1109/ICSE43902.2021.00098 10.1016/j.tics.2023.04.008 10.1109/ICSE-NIER58687.2023.00016 10.1145/3176349.3176893 10.1093/iwc/iwv046 10.1162/99608f92.1d3cf75d 10.1017/pan.2023.2 10.1145/3510460 10.1075/rs.18007.bib 10.2139/ssrn.4595896 10.1007/s10664-020-09858-z 10.1145/3544548.3580688 10.1145/3487193 10.1145/3581754.3584136 10.1109/MSR59073.2023.00088 |
| ContentType | Journal Article |
| Copyright | The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. |
| Copyright_xml | – notice: The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. |
| DBID | AAYXX CITATION 8FE 8FG ABJCF AFKRA ARAPS AZQEC BENPR BGLVJ CCPQU DWQXO GNUQQ HCIFZ JQ2 K7- L6V M7S P5Z P62 PHGZM PHGZT PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PTHSS |
| DOI | 10.1007/s10515-023-00409-6 |
| DatabaseName | CrossRef ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central UK/Ireland Advanced Technologies & Computer Science Collection ProQuest Central Essentials - QC ProQuest Central Technology collection ProQuest One Community College ProQuest Central ProQuest Central Student SciTech Premium Collection ProQuest Computer Science Collection Computer Science Database ProQuest Engineering Collection Engineering Database Advanced Technologies & Aerospace Database ProQuest Advanced Technologies & Aerospace Collection Proquest Central Premium ProQuest One Academic ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
| DatabaseTitle | CrossRef Computer Science Database ProQuest Central Student Technology Collection ProQuest One Academic Middle East (New) ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Essentials ProQuest Computer Science Collection SciTech Premium Collection ProQuest One Community College ProQuest Central China ProQuest Central ProQuest One Applied & Life Sciences ProQuest Engineering Collection ProQuest Central Korea ProQuest Central (New) Engineering Collection Advanced Technologies & Aerospace Collection Engineering Database ProQuest One Academic Eastern Edition ProQuest Technology Collection ProQuest SciTech Collection Advanced Technologies & Aerospace Database ProQuest One Academic UKI Edition Materials Science & Engineering Collection ProQuest One Academic ProQuest One Academic (New) |
| DatabaseTitleList | Computer Science Database |
| Database_xml | – sequence: 1 dbid: BENPR name: ProQuest Central url: https://www.proquest.com/central sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 1573-7535 |
| ExternalDocumentID | 10_1007_s10515_023_00409_6 |
| GrantInformation_xml | – fundername: NSF grantid: 2236198; 2235601; 2236198; 2235601 |
| GroupedDBID | -4Z -59 -5G -BR -EM -Y2 -~C .86 .DC .VR 06D 0R~ 0VY 199 1N0 1SB 2.D 203 23N 28- 2J2 2JN 2JY 2KG 2LR 2P1 2VQ 2~H 30V 4.4 406 408 409 40D 40E 5GY 5QI 5VS 67Z 6NX 78A 8TC 95- 95. 95~ 96X AAAVM AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AAOBN AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDZT ABECU ABFTD ABFTV ABHLI ABHQN ABJCF ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTEG ABTHY ABTKH ABTMW ABULA ABWNU ABXPI ACAOD ACBXY ACDTI ACGFS ACHSB ACHXU ACIWK ACKNC ACMDZ ACMLO ACOKC ACOMO ACPIV ACSNA ACZOJ ADHHG ADHIR ADIMF ADINQ ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF ADZKW AEBTG AEFIE AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFEXP AFGCZ AFKRA AFLOW AFQWF AFWTZ AFZKB AGAYW AGDGC AGGDS AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMXSW AMYLF AMYQR AOCGG ARAPS ARMRJ ASPBG AVWKF AXYYD AYJHY AZFZN B-. BA0 BBWZM BDATZ BENPR BGLVJ BGNMA BSONS CAG CCPQU COF CS3 CSCUP DDRTE DL5 DNIVK DPUIP EBLON EBS EIOEI EJD ESBYG FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNWQR GQ6 GQ7 GQ8 GXS H13 HCIFZ HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ I09 IHE IJ- IKXTQ ITM IWAJR IXC IZIGR IZQ I~X J-C J0Z JBSCW JCJTX JZLTJ K7- KDC KOV KOW LAK LLZTM M4Y M7S MA- MVM N2Q NB0 NDZJH NPVJJ NQJWS NU0 O9- O93 O9G O9I O9J OAM OVD P19 P2P P9O PF0 PT4 PT5 PTHSS QOK QOS R4E R89 R9I RHV RNI RNS ROL RPX RSV RZC RZE RZK S16 S1Z S26 S27 S28 S3B SAP SCJ SCLPG SCO SDH SDM SHX SISQX SJYHP SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 T16 TEORI TSG TSK TSV TUC U2A UG4 UOJIU UTJUX UZXMN VC2 VFIZW W23 W48 WK8 YLTOR Z45 Z7R Z7X Z83 Z88 Z8M Z8R Z8W Z92 ZMTXR ~A9 ~EX AAPKM AAYXX ABBRH ABDBE ABFSG ABRTQ ACSTC ADHKG AEZWR AFDZB AFFHD AFHIU AFOHR AGQPQ AHPBZ AHWEU AIXLP ATHPR AYFIA CITATION PHGZM PHGZT PQGLB 8FE 8FG AZQEC DWQXO GNUQQ JQ2 L6V P62 PKEHL PQEST PQQKQ PQUKI PRINS |
| ID | FETCH-LOGICAL-c319t-ff96f2facbea51c10db55ef56020f68a5ab32f6172a5fd80ecfe8ec61b3740e43 |
| IEDL.DBID | K7- |
| ISICitedReferencesCount | 10 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001140056400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0928-8910 |
| IngestDate | Thu Nov 06 14:30:27 EST 2025 Tue Nov 18 21:51:53 EST 2025 Sat Nov 29 02:06:33 EST 2025 Fri Feb 21 02:39:44 EST 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Keywords | Qualitative research Foundation models Large language models Software engineering |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c319t-ff96f2facbea51c10db55ef56020f68a5ab32f6172a5fd80ecfe8ec61b3740e43 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| PQID | 3048256396 |
| PQPubID | 2043871 |
| ParticipantIDs | proquest_journals_3048256396 crossref_primary_10_1007_s10515_023_00409_6 crossref_citationtrail_10_1007_s10515_023_00409_6 springer_journals_10_1007_s10515_023_00409_6 |
| PublicationCentury | 2000 |
| PublicationDate | 20240500 2024-05-00 20240501 |
| PublicationDateYYYYMMDD | 2024-05-01 |
| PublicationDate_xml | – month: 5 year: 2024 text: 20240500 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York – name: Dordrecht |
| PublicationSubtitle | An International Journal |
| PublicationTitle | Automated software engineering |
| PublicationTitleAbbrev | Autom Softw Eng |
| PublicationYear | 2024 |
| Publisher | Springer US Springer Nature B.V |
| Publisher_xml | – name: Springer US – name: Springer Nature B.V |
| References | Smith, M., Danilova, A., Naiakshina, A.: A meta-research agenda for recruitment and study design for developer studies. In: 1st International Workshop on Recruiting Participants for Empirical Software Engineering (RoPES’22), 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE) (2022) ArgyleLPBusbyECFuldaNGublerJRRyttingCWingateDOut of one, many: using language models to simulate human samplesPolit. Anal.202331333735110.1017/pan.2023.2 Treude, C., Hata, H.: She elicits requirements and he tests: software engineering gender bias in large language models (2023). arXiv:2303.10131 TrinkenreichBWieseISarmaAGerosaMSteinmacherIWomen’s participation in open source software: a survey of the literatureACM Trans. Softw. Eng. Methodol. (TOSEM)202231413710.1145/3510460 HutsonMMastinAGuinea pigbotsScience (New York, NY)2023381665412112310.1126/science.adj6791 Kaddour, J., Harris, J., Mozes, M., Bradley, H., Raileanu, R., McHardy, R.: Challenges and applications of large language models (2023). arXiv:2307.10169 Lee, S., Peng, T.-Q., Goldberg, M.H., Rosenthal, S.A., Kotcher, J.E., Maibach, E.W., Leiserowitz, A.: Can large language models capture public opinion about global warming? An empirical assessment of algorithmic fidelity and bias (2023). arXiv:2311.00217 Simmons, G., Hare, C.: Large language models as subpopulation representative models: a review (2023). arXiv:2310.17888 DemszkyDYangDYeagerDSBryanCJClapperMChandhokSEichstaedtJCHechtCJamiesonJJohnsonMUsing large language models in psychologyNat. Rev. Psychol.20232114 Hämäläinen, P., Tavast, M., Kunnari, A.: Evaluating large language models in generating synthetic HCI research data: a case study. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. CHI ’23. Association for Computing Machinery, New York, NY, USA (2023). https://doi.org/10.1145/3544548.3580688 Eliot, L.: The bold promise Of mega-personas as a new shake-up for prompt engineering generative AI techniques (2023). Accessed 08 Nov 2023. https://www.forbes.com/sites/lanceeliot/2023/08/15/the-bold-promise-of-mega-personas-as-a-new-shake-up-for-prompt-engineering-generative-ai-techniques/?sh=2be155065552 BiberDText-linguistic approaches to register variationRegist. Stud.201911427510.1075/rs.18007.bib StoreyM-AErnstNAWilliamsCKalliamvakouEThe who, what, how of software engineering research: a socio-technical frameworkEmpir. Softw. Eng.2020254097412910.1007/s10664-020-09858-z Gerosa, M., Wiese, I., Trinkenreich, B., Link, G., Robles, G., Treude, C., Steinmacher, I., Sarma, A.: The shifting sands of motivation: Revisiting what drives contributors in open source. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pp. 1046–1058. IEEE (2021) Sanders, N.E., Ulinich, A., Schneier, B.: Demonstrations of the potential of AI-based political issue polling (2023). arXiv:2307.04781 Suguri Motoki, F.Y., Monteiro, J., Malagueño, R., Rodrigues, V.: From data scarcity to data abundance: crafting synthetic survey data in management accounting using ChatGPT (2023). Available at SSRN DillionDTandonNGuYGrayKCan AI language models replace human participants?Trends Cogn. Sci.202327759760010.1016/j.tics.2023.04.008 Chew, R., Bollenbacher, J., Wenger, M., Speer, J., Kim, A.: LLM-assisted content analysis: using large language models to support deductive coding (2023). arXiv:2306.14924 Dai, S.-C., Xiong, A., Ku, L.-W.: LLM-in-the-loop: leveraging large language model for thematic analysis (2023). arXiv:2310.15100 ChavesAPEgbertJHockingTDoerryEGerosaMAChatbots language design: the influence of language variation on user experience with tourist assistant chatbotsACM Trans. Comput. Hum. Interact.202229213810.1145/3487193 Xiao, Z., Yuan, X., Liao, Q.V., Abdelghani, R., Oudeyer, P.-Y.: Supporting qualitative analysis with large language models: combining codebook with GPT-3 for deductive coding. In: Companion Proceedings of the 28th International Conference on Intelligent User Interfaces, pp. 75–78 (2023) Kim, J., Lee, B.: AI-augmented surveys: leveraging large language models for opinion prediction in nationally representative surveys (2023). arXiv:2305.09620 BurnettMStumpfSMacbethJMakriSBeckwithLKwanIPetersAJerniganWGendermag: a method for evaluating software’s gender inclusivenessInteract. Comput.201628676078710.1093/iwc/iwv046 De Paoli, S.: Improved prompting and process for writing user personas with LLMs, using qualitative interviews: capturing behaviour and personality traits of users (2023). arXiv:2310.06391 Aher, G.V., Arriaga, R.I., Kalai, A.T.: Using large language models to simulate multiple humans and replicate human subject studies. In: International Conference on Machine Learning, pp. 337–371. PMLR (2023) Kokinda, E., Moster, M., Dominic, J., Rodeghero, P.: Under the bridge: trolling and the challenges of recruiting software developers for empirical research studies. In: 2023 IEEE/ACM 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), pp. 55–59 (2023). https://doi.org/10.1109/ICSE-NIER58687.2023.00016 Jiang, H., Zhang, X., Cao, X., Kabbara, J., Roy, D.: PersonaLLM: investigating the ability of GPT-3.5 to express personality traits and gender differences (2023). arXiv:2305.02547 Zhou, Y., Muresanu, A.I., Han, Z., Paster, K., Pitis, S., Chan, H., Ba, J.: Large language models are human-level prompt engineers (2022). arXiv:2211.01910 Jung, S.-g., Salminen, J., Kwak, H., An, J., Jansen, B.J.: Automatic persona generation (APG) a rationale and demonstration. In: Proceedings of the 2018 Conference on Human Information Interaction and Retrieval, pp. 321–324 (2018) Wang, Z., Mao, S., Wu, W., Ge, T., Wei, F., Ji, H.: Unleashing cognitive synergy in large language models: a task-solving agent through multi-persona self-collaboration (2023). arXiv:2307.05300 LP Argyle (409_CR2) 2023; 31 409_CR30 M Burnett (409_CR4) 2016; 28 409_CR16 409_CR17 409_CR15 409_CR12 409_CR13 409_CR11 409_CR6 409_CR7 409_CR8 409_CR1 D Demszky (409_CR9) 2023; 2 409_CR18 409_CR19 D Biber (409_CR3) 2019; 1 M-A Storey (409_CR24) 2020; 25 409_CR20 B Trinkenreich (409_CR27) 2022; 31 M Hutson (409_CR14) 2023; 381 D Dillion (409_CR10) 2023; 27 409_CR28 409_CR25 409_CR26 409_CR23 409_CR21 409_CR22 AP Chaves (409_CR5) 2022; 29 409_CR29 |
| References_xml | – reference: DemszkyDYangDYeagerDSBryanCJClapperMChandhokSEichstaedtJCHechtCJamiesonJJohnsonMUsing large language models in psychologyNat. Rev. Psychol.20232114 – reference: StoreyM-AErnstNAWilliamsCKalliamvakouEThe who, what, how of software engineering research: a socio-technical frameworkEmpir. Softw. Eng.2020254097412910.1007/s10664-020-09858-z – reference: BiberDText-linguistic approaches to register variationRegist. Stud.201911427510.1075/rs.18007.bib – reference: Wang, Z., Mao, S., Wu, W., Ge, T., Wei, F., Ji, H.: Unleashing cognitive synergy in large language models: a task-solving agent through multi-persona self-collaboration (2023). arXiv:2307.05300 – reference: Simmons, G., Hare, C.: Large language models as subpopulation representative models: a review (2023). arXiv:2310.17888 – reference: ArgyleLPBusbyECFuldaNGublerJRRyttingCWingateDOut of one, many: using language models to simulate human samplesPolit. Anal.202331333735110.1017/pan.2023.2 – reference: De Paoli, S.: Improved prompting and process for writing user personas with LLMs, using qualitative interviews: capturing behaviour and personality traits of users (2023). arXiv:2310.06391 – reference: Smith, M., Danilova, A., Naiakshina, A.: A meta-research agenda for recruitment and study design for developer studies. In: 1st International Workshop on Recruiting Participants for Empirical Software Engineering (RoPES’22), 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE) (2022) – reference: BurnettMStumpfSMacbethJMakriSBeckwithLKwanIPetersAJerniganWGendermag: a method for evaluating software’s gender inclusivenessInteract. Comput.201628676078710.1093/iwc/iwv046 – reference: Chew, R., Bollenbacher, J., Wenger, M., Speer, J., Kim, A.: LLM-assisted content analysis: using large language models to support deductive coding (2023). arXiv:2306.14924 – reference: Lee, S., Peng, T.-Q., Goldberg, M.H., Rosenthal, S.A., Kotcher, J.E., Maibach, E.W., Leiserowitz, A.: Can large language models capture public opinion about global warming? An empirical assessment of algorithmic fidelity and bias (2023). arXiv:2311.00217 – reference: TrinkenreichBWieseISarmaAGerosaMSteinmacherIWomen’s participation in open source software: a survey of the literatureACM Trans. Softw. Eng. Methodol. (TOSEM)202231413710.1145/3510460 – reference: Treude, C., Hata, H.: She elicits requirements and he tests: software engineering gender bias in large language models (2023). arXiv:2303.10131 – reference: Dai, S.-C., Xiong, A., Ku, L.-W.: LLM-in-the-loop: leveraging large language model for thematic analysis (2023). arXiv:2310.15100 – reference: Sanders, N.E., Ulinich, A., Schneier, B.: Demonstrations of the potential of AI-based political issue polling (2023). arXiv:2307.04781 – reference: Aher, G.V., Arriaga, R.I., Kalai, A.T.: Using large language models to simulate multiple humans and replicate human subject studies. In: International Conference on Machine Learning, pp. 337–371. PMLR (2023) – reference: Gerosa, M., Wiese, I., Trinkenreich, B., Link, G., Robles, G., Treude, C., Steinmacher, I., Sarma, A.: The shifting sands of motivation: Revisiting what drives contributors in open source. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pp. 1046–1058. IEEE (2021) – reference: Kokinda, E., Moster, M., Dominic, J., Rodeghero, P.: Under the bridge: trolling and the challenges of recruiting software developers for empirical research studies. In: 2023 IEEE/ACM 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), pp. 55–59 (2023). https://doi.org/10.1109/ICSE-NIER58687.2023.00016 – reference: DillionDTandonNGuYGrayKCan AI language models replace human participants?Trends Cogn. Sci.202327759760010.1016/j.tics.2023.04.008 – reference: Xiao, Z., Yuan, X., Liao, Q.V., Abdelghani, R., Oudeyer, P.-Y.: Supporting qualitative analysis with large language models: combining codebook with GPT-3 for deductive coding. In: Companion Proceedings of the 28th International Conference on Intelligent User Interfaces, pp. 75–78 (2023) – reference: ChavesAPEgbertJHockingTDoerryEGerosaMAChatbots language design: the influence of language variation on user experience with tourist assistant chatbotsACM Trans. Comput. Hum. Interact.202229213810.1145/3487193 – reference: Kim, J., Lee, B.: AI-augmented surveys: leveraging large language models for opinion prediction in nationally representative surveys (2023). arXiv:2305.09620 – reference: Suguri Motoki, F.Y., Monteiro, J., Malagueño, R., Rodrigues, V.: From data scarcity to data abundance: crafting synthetic survey data in management accounting using ChatGPT (2023). Available at SSRN – reference: Zhou, Y., Muresanu, A.I., Han, Z., Paster, K., Pitis, S., Chan, H., Ba, J.: Large language models are human-level prompt engineers (2022). arXiv:2211.01910 – reference: Kaddour, J., Harris, J., Mozes, M., Bradley, H., Raileanu, R., McHardy, R.: Challenges and applications of large language models (2023). arXiv:2307.10169 – reference: Eliot, L.: The bold promise Of mega-personas as a new shake-up for prompt engineering generative AI techniques (2023). Accessed 08 Nov 2023. https://www.forbes.com/sites/lanceeliot/2023/08/15/the-bold-promise-of-mega-personas-as-a-new-shake-up-for-prompt-engineering-generative-ai-techniques/?sh=2be155065552 – reference: Hämäläinen, P., Tavast, M., Kunnari, A.: Evaluating large language models in generating synthetic HCI research data: a case study. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. CHI ’23. Association for Computing Machinery, New York, NY, USA (2023). https://doi.org/10.1145/3544548.3580688 – reference: HutsonMMastinAGuinea pigbotsScience (New York, NY)2023381665412112310.1126/science.adj6791 – reference: Jiang, H., Zhang, X., Cao, X., Kabbara, J., Roy, D.: PersonaLLM: investigating the ability of GPT-3.5 to express personality traits and gender differences (2023). arXiv:2305.02547 – reference: Jung, S.-g., Salminen, J., Kwak, H., An, J., Jansen, B.J.: Automatic persona generation (APG) a rationale and demonstration. In: Proceedings of the 2018 Conference on Human Information Interaction and Retrieval, pp. 321–324 (2018) – ident: 409_CR8 – volume: 381 start-page: 121 issue: 6654 year: 2023 ident: 409_CR14 publication-title: Science (New York, NY) doi: 10.1126/science.adj6791 – ident: 409_CR7 doi: 10.18653/v1/2023.findings-emnlp.669 – ident: 409_CR12 doi: 10.1109/ICSE43902.2021.00098 – ident: 409_CR20 – volume: 27 start-page: 597 issue: 7 year: 2023 ident: 409_CR10 publication-title: Trends Cogn. Sci. doi: 10.1016/j.tics.2023.04.008 – ident: 409_CR19 doi: 10.1109/ICSE-NIER58687.2023.00016 – ident: 409_CR22 – ident: 409_CR16 doi: 10.1145/3176349.3176893 – ident: 409_CR28 – volume: 28 start-page: 760 issue: 6 year: 2016 ident: 409_CR4 publication-title: Interact. Comput. doi: 10.1093/iwc/iwv046 – ident: 409_CR21 doi: 10.1162/99608f92.1d3cf75d – volume: 31 start-page: 337 issue: 3 year: 2023 ident: 409_CR2 publication-title: Polit. Anal. doi: 10.1017/pan.2023.2 – volume: 31 start-page: 1 issue: 4 year: 2022 ident: 409_CR27 publication-title: ACM Trans. Softw. Eng. Methodol. (TOSEM) doi: 10.1145/3510460 – ident: 409_CR6 – volume: 2 start-page: 1 year: 2023 ident: 409_CR9 publication-title: Nat. Rev. Psychol. – volume: 1 start-page: 42 issue: 1 year: 2019 ident: 409_CR3 publication-title: Regist. Stud. doi: 10.1075/rs.18007.bib – ident: 409_CR18 – ident: 409_CR25 doi: 10.2139/ssrn.4595896 – volume: 25 start-page: 4097 year: 2020 ident: 409_CR24 publication-title: Empir. Softw. Eng. doi: 10.1007/s10664-020-09858-z – ident: 409_CR13 doi: 10.1145/3544548.3580688 – ident: 409_CR23 – ident: 409_CR1 – volume: 29 start-page: 1 issue: 2 year: 2022 ident: 409_CR5 publication-title: ACM Trans. Comput. Hum. Interact. doi: 10.1145/3487193 – ident: 409_CR15 – ident: 409_CR11 – ident: 409_CR29 doi: 10.1145/3581754.3584136 – ident: 409_CR30 – ident: 409_CR26 doi: 10.1109/MSR59073.2023.00088 – ident: 409_CR17 |
| SSID | ssj0009699 |
| Score | 2.3950908 |
| Snippet | Research within sociotechnical domains, such as software engineering, fundamentally requires the human perspective. Nevertheless, traditional qualitative data... |
| SourceID | proquest crossref springer |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 13 |
| SubjectTerms | Artificial Intelligence Automation Collaboration Computer Science Data collection Documentation Engineering research Focus groups Human factors research Human subjects Interviews Language Large language models Qualitative analysis Qualitative research Researchers Software engineering Software Engineering/Programming and Operating Systems Women |
| SummonAdditionalLinks | – databaseName: SpringerLINK Contemporary 1997-Present dbid: RSV link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1JS8QwFA46evDiuOLoKDl400DadElOMgwOCjK4M7eSpi8oSJXpqH_fly5WRQW9lWahvLzla5L3PUL2sxB_ZEWUMW14zALfes6kODMicGxZQoRQsuufxeOxnEzUeZ0UVjS33ZsjydJTf0h2w9jLMMYwp3mKRfNkAcOddOZ4eXXbUu1GqmLY8yWTGA3rVJnv5_gcjlqM-eVYtIw2o-7_vnOFLNfokg4qdVglc5CvkW5TuYHWhrxOLoY6p4NT6rZkgeqCalqgB6muDVDEsbSs3edeun2agt7jMzrsVz0FCi2FIa25gu6ONsjN6Ph6eMLq4gq4DJ6aMWtVZH2rTQo69IzHszQMwSIA8rmNpA51Knzr8I0ObSY5GAsSTOSlIg44BGKTdPLHHLYITWWgU4vi5lDSoWmlLADnJlVSmjjuEa-RcWJq5nFXAOMhaTmTncwSlFlSyiyJeuTgfcxTxbvxa-9-s3RJbYNFItA5IaATCpsPm6Vqm3-ebftv3XfIko9Ip7oF2Sed2fQZdsmieZndF9O9UjffAB7f2_g priority: 102 providerName: Springer Nature |
| Title | Can AI serve as a substitute for human subjects in software engineering research? |
| URI | https://link.springer.com/article/10.1007/s10515-023-00409-6 https://www.proquest.com/docview/3048256396 |
| Volume | 31 |
| WOSCitedRecordID | wos001140056400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVPQU databaseName: Advanced Technologies & Aerospace Database customDbUrl: eissn: 1573-7535 dateEnd: 20241207 omitProxy: false ssIdentifier: ssj0009699 issn: 0928-8910 databaseCode: P5Z dateStart: 19970101 isFulltext: true titleUrlDefault: https://search.proquest.com/hightechjournals providerName: ProQuest – providerCode: PRVPQU databaseName: Computer Science Database customDbUrl: eissn: 1573-7535 dateEnd: 20241207 omitProxy: false ssIdentifier: ssj0009699 issn: 0928-8910 databaseCode: K7- dateStart: 19970101 isFulltext: true titleUrlDefault: http://search.proquest.com/compscijour providerName: ProQuest – providerCode: PRVPQU databaseName: Engineering Database customDbUrl: eissn: 1573-7535 dateEnd: 20241207 omitProxy: false ssIdentifier: ssj0009699 issn: 0928-8910 databaseCode: M7S dateStart: 19970101 isFulltext: true titleUrlDefault: http://search.proquest.com providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 1573-7535 dateEnd: 20241207 omitProxy: false ssIdentifier: ssj0009699 issn: 0928-8910 databaseCode: BENPR dateStart: 19970101 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVAVX databaseName: SpringerLINK Contemporary 1997-Present customDbUrl: eissn: 1573-7535 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0009699 issn: 0928-8910 databaseCode: RSV dateStart: 19970101 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8QwEB509eDF9Ymr65KDNw2m7_Qkq7goyrK6KuKlpGmCC1J1u-rfd9KmFgW9eAmlj1DyTWYmk8k3AHtZgAtZL8yokCyivqsdM6UYlZ5v2LI8L1Alu_5lNBzy-_t4ZANuhU2rrHViqaizZ2li5Ie47MbFDNrT8OjllZqqUWZ31ZbQmIcFx3UdI-cXEW1Id8O44tpzOeVoF-2hGXt0Di05RYtFjRzHNPxumBpv88cGaWl3Bu3__vEKLFuPk_QrEVmFOZWvQbuu5kDs5F6HqxORk_45MWFaRURBBClQq1SpBAR9W1LW8zM3TeymIBO8RiX-IaaKqIbWkFj-oMejDbgdnN6cnFFbcAGhceIZ1ToOtauFTJUIHOmwLA0CpdEpcpkOuQhE6rna-Dwi0BlnSmrFlQyd1It8pnxvE1r5c662gKTcF6nGgWeqpEgTcayVYkymMecyijrg1KOdSMtGbopiPCUNj7JBKEGEkhKhJOzA_tc3LxUXx59vd2tYEjsvi6TBpAMHNbDN49972_67tx1YctHbqTIhu9CaTd_ULizK99mkmPZg4fh0OLruldLZM-mlY2xHwQO21-O7TxEK6oI |
| linkProvider | ProQuest |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Lb9NAEB6VgEQvlEerBlrYA5zKquv1a32ooqhQNWqIQBQpN7Nez4pKyEnjtFH_FL-xs37UohK95cDN8mMte7-Zb2Yf3wC8z0NKZP0o59qImAfSes6kBDd-4NSyfD_ESl1_HE8majpNvm7An3YvjFtW2frEylHnM-PGyA8p7aZkhvg0Gswvuasa5WZX2xIaNSzO8GZFKVt5NPpE_ftBypPP58envKkqQO_3kiW3NomstNpkqEPPeCLPwhAtMb8UNlI61JkvrSN2HdpcCTQWFZrIy_w4EBj41O4jeBw4718tFfzeifxGSa3tJxVXxMPNJp1mqx5FDpwYkju7SXj0NxF20e29CdmK5062_rc_9ByeNRE1G9Ym8AI2sHgJW221CtY4r1fw7VgXbDhibhgamS6ZZiV5zXqpBKPYnVX1Ct1JNzZVsgs6JpJa6QUy7GQbWaOP9GuwDT_W8mE70CtmBe4Cy1SgM0sdLbCSgNNJYhGFMFmilInjPnht76amUVt3RT9-p51OtENESohIK0SkUR8O7p6Z11ojD96918IgbfxOmXYY6MPHFkjd5X-39vrh1t7B09PzL-N0PJqcvYFNSZFdvepzD3rLxRXuwxNzvbwoF28ri2Dwc90AuwU0UkYR |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1bS8MwFA46RXxxXnE6NQ--aTBtekmeZEyHwzEmXthbSNMEB1LHWvXvm_Rip6ggvpUmDeUk55wvyTnfAeA49s1GlgQxEhKHyHO1Y1UKI0k8y5ZFiK9ydv1BOBzS8ZiN5rL482j36kqyyGmwLE1JdjaN9dlc4pvxw8j4G2RXIUPBIljybNEgu1-_fahpdwNWsO25FFHjGcu0me_H-Oyaarz55Yo09zy95v__eR2slagTdoplsgEWVLIJmlVFB1gq-Ba46YoEdvrQHtUqKFIoYGosSxFOAA2-hXlNP_vSnt-kcGKejSF_EzMFVU1tCEsOocfzbXDfu7zrXqGy6IKZHodlSGsWaFcLGSnhO9LBceT7Shtg5GIdUOGLiLja4h7h65hiJbWiSgZOREIPK4_sgEbynKhdACPqiUgb0WOV06QJxrRSGMuIUSrDsAWcSt5clozktjDGE6-5lK3MuJEZz2XGgxY4-fhmWvBx_Nq7XU0jL3Uz5cQYLQP0CDPNp9W01c0_j7b3t-5HYGV00eOD_vB6H6y6BgwVgZJt0MhmL-oALMvXbJLODvMl-w7VY-fA |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Can+AI+serve+as+a+substitute+for+human+subjects+in+software+engineering+research%3F&rft.jtitle=Automated+software+engineering&rft.au=Gerosa%2C+Marco&rft.au=Trinkenreich%2C+Bianca&rft.au=Steinmacher%2C+Igor&rft.au=Sarma%2C+Anita&rft.date=2024-05-01&rft.pub=Springer+Nature+B.V&rft.issn=0928-8910&rft.eissn=1573-7535&rft.volume=31&rft.issue=1&rft.spage=13&rft_id=info:doi/10.1007%2Fs10515-023-00409-6 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0928-8910&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0928-8910&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0928-8910&client=summon |