AI lifecycle models need to be revised An exploratory study in Fintech
Tech-leading organizations are embracing the forthcoming artificial intelligence revolution. Intelligent systems are replacing and cooperating with traditional software components. Thus, the same development processes and standards in software engineering ought to be complied in artificial intellige...
Saved in:
| Published in: | Empirical software engineering : an international journal Vol. 26; no. 5 |
|---|---|
| Main Authors: | , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
New York
Springer US
01.09.2021
|
| Subjects: | |
| ISSN: | 1382-3256, 1573-7616 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Tech-leading organizations are embracing the forthcoming artificial intelligence revolution. Intelligent systems are replacing and cooperating with traditional software components. Thus, the same development processes and standards in software engineering ought to be complied in artificial intelligence systems. This study aims to understand the processes by which artificial intelligence-based systems are developed and how state-of-the-art lifecycle models fit the current needs of the industry. We conducted an exploratory case study at ING, a global bank with a strong European base. We interviewed 17 people with different roles and from different departments within the organization. We have found that the following stages have been overlooked by previous lifecycle models:
data collection
,
feasibility study
,
documentation
,
model monitoring
, and
model risk assessment
. Our work shows that the real challenges of applying Machine Learning go much beyond sophisticated learning algorithms – more focus is needed on the entire lifecycle. In particular, regardless of the existing development tools for Machine Learning, we observe that they are still not meeting the particularities of this field. |
|---|---|
| AbstractList | Tech-leading organizations are embracing the forthcoming artificial intelligence revolution. Intelligent systems are replacing and cooperating with traditional software components. Thus, the same development processes and standards in software engineering ought to be complied in artificial intelligence systems. This study aims to understand the processes by which artificial intelligence-based systems are developed and how state-of-the-art lifecycle models fit the current needs of the industry. We conducted an exploratory case study at ING, a global bank with a strong European base. We interviewed 17 people with different roles and from different departments within the organization. We have found that the following stages have been overlooked by previous lifecycle models:
data collection
,
feasibility study
,
documentation
,
model monitoring
, and
model risk assessment
. Our work shows that the real challenges of applying Machine Learning go much beyond sophisticated learning algorithms – more focus is needed on the entire lifecycle. In particular, regardless of the existing development tools for Machine Learning, we observe that they are still not meeting the particularities of this field. |
| ArticleNumber | 95 |
| Author | Haakman, Mark van Deursen, Arie Cruz, Luís Huijgens, Hennie |
| Author_xml | – sequence: 1 givenname: Mark surname: Haakman fullname: Haakman, Mark organization: AI For Fintech Research, ING – sequence: 2 givenname: Luís orcidid: 0000-0002-1615-355X surname: Cruz fullname: Cruz, Luís email: l.cruz@tudelft.nl organization: Delft University of Technology – sequence: 3 givenname: Hennie surname: Huijgens fullname: Huijgens, Hennie organization: AI For Fintech Research, ING – sequence: 4 givenname: Arie surname: van Deursen fullname: van Deursen, Arie organization: Delft University of Technology |
| BookMark | eNp9zz1PwzAQgGELFYm28AeYMrEZzl9xPFYV0EqVWGC2_HFGqdIE2QGp_55AmRg63Q33nPQuyKwfeiTklsE9A9APhUFdSwqcUTDGCMouyJwpLaiuWT2bdtFwKriqr8iilD0AGC3VnNyttlXXJgzH0GF1GCJ2peoRYzUOlccq41dbMF6Ty-S6gjd_c0nenh5f1xu6e3nerlc7GrhhI00-Bs8h8iSk99qlKJWUOiloIkPJQUceagkGIUSnE3qlvNFMJ6l0Y5xYkub0N-ShlIzJhnZ0Yzv0Y3ZtZxnYn1576rVTr_3ttWyi_B_9yO3B5eN5JE6oTMf9O2a7Hz5zPyWeU9_Sl2kn |
| CitedBy_id | crossref_primary_10_1007_s00415_025_12918_3 crossref_primary_10_1109_MC_2023_3254646 crossref_primary_10_1002_widm_1507 crossref_primary_10_3389_fdata_2024_1467222 crossref_primary_10_1145_3688841 crossref_primary_10_3233_NAI_240767 crossref_primary_10_1080_12460125_2023_2214306 crossref_primary_10_1016_j_sftr_2025_101279 crossref_primary_10_1590_0001_3765202520240924 crossref_primary_10_3390_technologies13090421 crossref_primary_10_1016_j_datak_2022_102013 crossref_primary_10_1016_j_eneco_2024_107561 crossref_primary_10_1080_12460125_2025_2537085 crossref_primary_10_1016_j_soncn_2023_151433 crossref_primary_10_1145_3625289 crossref_primary_10_3390_ai4040046 crossref_primary_10_1016_j_irfa_2023_102800 crossref_primary_10_1145_3702004 crossref_primary_10_1007_s12599_023_00842_7 crossref_primary_10_32517_0234_0453_2023_38_3_31_41 crossref_primary_10_1515_zwf_2022_1107 crossref_primary_10_1109_ACCESS_2025_3577970 crossref_primary_10_1007_s43621_025_01225_6 crossref_primary_10_1073_pnas_2204781120 crossref_primary_10_1016_j_procir_2024_10_279 crossref_primary_10_1145_3712007 crossref_primary_10_1186_s41120_025_00110_w crossref_primary_10_1016_j_imu_2024_101472 crossref_primary_10_1016_j_infsof_2024_107520 crossref_primary_10_1080_17517575_2025_2481557 crossref_primary_10_1109_TAI_2023_3315272 crossref_primary_10_1051_shsconf_202420301005 crossref_primary_10_1007_s00146_024_01974_4 |
| Cites_doi | 10.1016/0147-1767(85)90062-8 10.1007/978-3-030-64148-1_12 10.14236/ewic/EASE2008.5 10.1007/978-3-030-00847-5_1 10.1109/VLHCC.2016.7739680 10.1145/2884781.2884833 10.1109/BigData.2017.8258038 10.1002/j.2334-5837.2017.00427.x 10.1007/978-3-030-58666-9_2 10.1007/978-1-4613-1639-8 10.1017/S0269888910000032 10.17763/haer.62.3.8323320856251826 10.1086/268415 10.4135/9781849209717 10.1109/SEAA.2018.00018 10.1145/3368089.3409711 10.1177/160940690600500107 10.1109/TSE.2017.2754374 10.1007/s10664-019-09682-0 10.1145/3292500.3330667 10.1109/ICSE-SEIP.2019.00042 10.1109/MS.2019.2954841 10.1006/jesp.1997.1333 10.1016/j.apnr.2005.06.001 10.1109/TSE.2019.2962027 10.1145/3287560.3287596 10.1109/HPCA.2019.00048 10.1145/3292500.3330744 10.2139/ssrn.3333515 10.1145/2481244.2481247 10.1145/3382494.3410681 10.1023/A:1025419826662 |
| ContentType | Journal Article |
| Copyright | The Author(s) 2021 |
| Copyright_xml | – notice: The Author(s) 2021 |
| DBID | C6C AAYXX CITATION |
| DOI | 10.1007/s10664-021-09993-1 |
| DatabaseName | Springer Nature OA Free Journals (WRLC) CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | CrossRef |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 1573-7616 |
| ExternalDocumentID | 10_1007_s10664_021_09993_1 |
| GroupedDBID | -4Z -59 -5G -BR -EM -Y2 -~C .86 .DC .VR 06D 0R~ 0VY 199 1N0 1SB 2.D 203 28- 29G 2J2 2JN 2JY 2KG 2LR 2P1 2VQ 2~H 30V 4.4 406 408 409 40D 40E 5GY 5QI 5VS 67Z 6NX 78A 8FE 8FG 8TC 8UJ 95- 95. 95~ 96X AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AAOBN AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYOK AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDZT ABECU ABFTD ABFTV ABHLI ABHQN ABJCF ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTEG ABTHY ABTKH ABTMW ABULA ABWNU ABXPI ACAOD ACBXY ACDTI ACGFS ACHSB ACHXU ACIWK ACKNC ACMDZ ACMLO ACOKC ACOMO ACPIV ACSNA ACZOJ ADHHG ADHIR ADIMF ADINQ ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF ADZKW AEBTG AEFIE AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFEXP AFGCZ AFKRA AFLOW AFQWF AFWTZ AFZKB AGAYW AGDGC AGGDS AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMXSW AMYLF AMYQR AOCGG ARAPS ARMRJ ASPBG AVWKF AXYYD AYJHY AZFZN B-. BA0 BBWZM BDATZ BENPR BGLVJ BGNMA BSONS C6C CAG CCPQU COF CS3 CSCUP DDRTE DL5 DNIVK DPUIP DU5 EBLON EBS EIOEI EJD ESBYG FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNWQR GQ6 GQ7 GQ8 GXS H13 HCIFZ HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ I09 IHE IJ- IKXTQ ITM IWAJR IXC IZIGR IZQ I~X I~Z J-C J0Z JBSCW JCJTX JZLTJ KDC KOV KOW L6V LAK LLZTM M4Y M7S MA- N2Q NB0 NDZJH NPVJJ NQJWS NU0 O9- O93 O9G O9I O9J OAM P19 P62 P9O PF0 PT4 PT5 PTHSS Q2X QOK QOS R4E R89 R9I RHV RNI RNS ROL RPX RSV RZC RZE RZK S0W S16 S1Z S26 S27 S28 S3B SAP SCJ SCLPG SCO SDH SDM SHX SISQX SJYHP SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 T16 TSG TSK TSV TUC U2A UG4 UOJIU UTJUX UZXMN VC2 VFIZW W23 W48 WK8 YLTOR Z45 Z7R Z7S Z7V Z7X Z7Z Z81 Z83 Z86 Z88 Z8M Z8N Z8P Z8R Z8T Z8U Z8W Z92 ZMTXR ~EX AAPKM AAYXX ABBRH ABDBE ABFSG ABRTQ ACSTC ADHKG AEZWR AFDZB AFFHD AFHIU AFOHR AGQPQ AHPBZ AHWEU AIXLP ATHPR AYFIA CITATION PHGZM PHGZT PQGLB |
| ID | FETCH-LOGICAL-c291t-fbdcb20d2f34bb7afd45447f508d1e4207d2c6409e0cda7feb55b9717f45789a3 |
| IEDL.DBID | RSV |
| ISICitedReferencesCount | 56 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000691590700003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1382-3256 |
| IngestDate | Tue Nov 18 22:24:57 EST 2025 Sat Nov 29 05:37:45 EST 2025 Fri Feb 21 02:48:46 EST 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 5 |
| Keywords | Case study AI engineering AI lifecycle SE4AI Machine learning |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c291t-fbdcb20d2f34bb7afd45447f508d1e4207d2c6409e0cda7feb55b9717f45789a3 |
| ORCID | 0000-0002-1615-355X |
| OpenAccessLink | https://link.springer.com/10.1007/s10664-021-09993-1 |
| ParticipantIDs | crossref_citationtrail_10_1007_s10664_021_09993_1 crossref_primary_10_1007_s10664_021_09993_1 springer_journals_10_1007_s10664_021_09993_1 |
| PublicationCentury | 2000 |
| PublicationDate | 20210900 2021-09-00 |
| PublicationDateYYYYMMDD | 2021-09-01 |
| PublicationDate_xml | – month: 9 year: 2021 text: 20210900 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationSubtitle | An International Journal |
| PublicationTitle | Empirical software engineering : an international journal |
| PublicationTitleAbbrev | Empir Software Eng |
| PublicationYear | 2021 |
| Publisher | Springer US |
| Publisher_xml | – name: Springer US |
| References | Rollins J (2015) Foundational methodology for data science. Domino Data Lab, Inc. Whitepaper Ryan M J, Wheatcraft L S (2017) On the use of the terms verification and validation. In: INCOSE International Symposium, vol 27, pp 1277–1290. https://doi.org/10.1002/j.2334-5837.2017.00427.x YinRKCase study research and applications: Design and methods2017Thousand OaksSage publications Ericson G, Rohm WA, Martens J, Sharkey K, Casey C, Harvey B, Schonning N (2017) Team data science process documentation. Retrieved September 2020 https://docs.microsoft.com/en-us/azure/machine-learning/team-data-science-process Martínez-Plumed F, Contreras-Ochando L, Ferri C, Flach P, Hernández-Orallo J, Kull M, Lachiche N, Ramírez-Quintana MJ (2017) Casp-dm: Context aware standard process for data mining. arXiv:1709.09003 MeadCIsmailMAnalog VLSI implementation of neural systems1989BerlinSpringer10.1007/978-1-4613-1639-8 Wu C J, Brooks D, Chen K, Chen D, Choudhury S, Dukhan M, Hazelwood K, Isaac E, Jia Y, Jia B et al (2019) Machine learning at facebook: Understanding inference at the edge, IEEE Martínez-Plumed F, Contreras-Ochando L, Ferri C, Orallo J H, Kull M, Lachiche N, Quintana M J R, Flach PA (2019) CRISP-DM twenty years later: From data mining processes to data science trajectories. IEEE Trans Knowl Data Eng Moyle S, Jorge A (2001) Ramsys-a methodology for supporting rapid remote collaborative data mining projects, vol 64 LinJRyaboyDScaling big data mining infrastructure: the Twitter experienceAcm SIGKDD Explorations Newsletter201314261910.1145/2481244.2481247 Breck E, Cai S, Nielsen E, Salib M, Sculley D (2017) The ml test score: A rubric for ml production readiness and technical debt reduction, IEEE ShearerCThe CRISP-DM model: the new blueprint for data miningJ Data Warehous2000541322 WangYCognitive informatics: A new transdisciplinary research fieldBrain and Mind20034211512710.1023/A:1025419826662https://doi.org/10.1023/A:1025419826662 MariscalGMarbanOFernandezCA survey of data mining and knowledge discovery process models and methodologiesKnowl Eng Rev201025213716610.1017/S0269888910000032 WuXZhuXWuGQDingWData mining with big dataIEEE Trans Knowl Data Eng201326197107 Amershi S, Begel A, Bird C, DeLine R, Gall H, Kamar E, Nagappan N, Nushi B, Zimmermann T (2019) Software engineering for machine learning: a case study. In: Proceedings of the 41st International conference on software engineering, software engineering in practice. IEEE Press, pp 291–300 BrummerCYadavYFintech and the innovation trilemmaGeo LJ2018107235 Damiani E, Frati F (2018) Towards conceptual models for machine learning computations. In: International conference on conceptual modeling. Springer, pp 3–9 ING (2019) ING at a glance. https://www.ing.com/About-us/Profile/ING-at-a-glance.htm FeredayJMuir-CochraneEDemonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme developmentInt J Qual Methods200651809210.1177/160940690600500107 Haakman M, Cruz L, Huijgens H, van Deursen A (2020) Machine learning behind the scenes: An exploratory study in fintech - case study protocol. https://1drv.ms/b/s!AuvX_CBP4YARcBBYdYAOG8qLGIk MenziesTThe five laws of se for aiIEEE Softw.2019371818510.1109/MS.2019.2954841 Mitchell M, Wu S, Zaldivar A, Barnes P, Vasserman L, Hutchinson B, Spitzer E, Raji ID, Gebru T (2019) Model cards for model reporting. In: Proceedings of the conference on fairness, accountability, and transparency, pp 220–229 15288 I S O /I E C /I E E E (2015) Systems and software engineering – System life cycle processes. Institute of Electrical and Electronic Engineers (IEEE), New York Van LooRMaking innovation more competitive: the case of fintechUCLA L Rev201865232 Lwakatare LE, Crnkovic I, Rånge E, Bosch J (2020) From a data science driven process to a continuous delivery process for machine learning systems. In: International conference on product-focused software process improvement. Springer, pp 185–201 Brereton P, Kitchenham B A, Budgen D, Li Z (2008) Using a protocol template for case study planning. In: EASE, Citeseer, vol 8, pp 41–48 Hill C, Bellamy R, Erickson T, Burnett M (2016) Trials and tribulations of developers of intelligent systems: A field study. In: 2016 IEEE symposium on visual languages and human-centric computing (VL/HCC). IEEE, pp 162–170 Kavuri AS, Milne A (2019) Fintech and the future of financial services: What are the research gaps CAMA Working Paper No 18/2019 MitchellTRThompsonLPetersonECronkRTemporal adjustments in the evaluation of events: The “rosy view”J Exper Soc Psychol199733442144810.1006/jesp.1997.1333 Sculley D, Holt G, Golovin D, Davydov E, Phillips T, Ebner D, Chaudhary V, Young M, Crespo JF, Dennison D (2015) Hidden technical debt in machine learning systems. In: Advances in neural information processing systems, pp 2503–2511 Akkiraju R, Sinha V, Xu A, Mahmud J, Gundecha P, Liu Z, Liu X, Schumacher J (2020) Characterizing machine learning processes: A maturity framework. In: International conference on business process management, Springer, pp 17–31 Cruz L, Abreu R (2019) Catalog of energy patterns for mobile applications. Empir Softw Eng. https://doi.org/10.1007/s10664-019-09682-0 StraussACorbinJBasics of qualitative research1990Thousand OaksSage publications HalcombEJDavidsonPMIs verbatim transcription of interview data always necessary?Appl Nurs Res2006191384210.1016/j.apnr.2005.06.001 Arpteg A, Brinne B, Crnkovic-Friis L, Bosch J (2018) Software engineering challenges of deep learning, IEEE Larios Vargas E, Aniche M, Treude C, Bruntink M, Gousios G (2020) Selecting third-party libraries: The practitioners’ perspective. In: Proceedings of the 28th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp 245–256 Serban A, van der Blom K, Hoos H, Visser J (2020) Adoption and effects of software engineering best practices in machine learning. In: Empirical software engineering and measurement MaxwellJUnderstanding and validity in qualitative researchHarvard Educ Rev199262327930110.17763/haer.62.3.8323320856251826 Ahmed Z, Amizadeh S, Bilenko M, Carr R, Chin WS, Dekel Y, Dupre X, Eksarevskiy V, Filipi S, Finley T et al (2019) Machine learning at Microsoft with ML. NET. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. pp 2448–2458 Stol KJ, Ralph P, Fitzgerald B (2016) Grounded theory in software engineering research: a critical review and guidelines. In: Proceedings of the 38th International conference on software engineering, pp 120–131 KimMZimmermannTDeLineRBegelAData scientists in software teams: State of the art and challengesIEEE Trans. Softw. Eng.201744111024103810.1109/TSE.2017.2754374 Zhang J M, Harman M, Ma L (2020) Liu Y, Machine learning testing, Survey, landscapes and horizons. IEEE Trans Softw Eng Bernardi L, Mavridis T, Estevez P (2019) 150 successful machine learning models: 6 lessons learned at Booking.com. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. ACM, pp 1743–1751 FasickFASome uses of untranscribed tape recordings in survey researchPublic Opinion Q197741454955210.1086/268415 LincolnYGubaENaturalistic inquiry1985Newbury ParkSAGE10.1016/0147-1767(85)90062-8 WengrafTQualitative research interviewing: Biographic narrative and semi-structured methods2001Newbury ParkSAGE10.4135/9781849209717 Nicolescu B, Ertas A (2008) Transdisciplinary theory and practice. USA, TheATLAS 9993_CR9 TR Mitchell (9993_CR32) 1997; 33 T Menzies (9993_CR30) 2019; 37 9993_CR5 9993_CR4 T Wengraf (9993_CR44) 2001 9993_CR7 C Mead (9993_CR29) 1989 Y Wang (9993_CR43) 2003; 4 9993_CR6 J Fereday (9993_CR13) 2006; 5 C Shearer (9993_CR39) 2000; 5 RK Yin (9993_CR47) 2017 Y Lincoln (9993_CR23) 1985 J Lin (9993_CR22) 2013; 14 9993_CR45 9993_CR24 9993_CR40 9993_CR21 9993_CR26 9993_CR48 9993_CR27 J Maxwell (9993_CR28) 1992; 62 G Mariscal (9993_CR25) 2010; 25 A Strauss (9993_CR41) 1990 R Van Loo (9993_CR42) 2018; 65 EJ Halcomb (9993_CR15) 2006; 19 9993_CR11 9993_CR33 9993_CR34 9993_CR35 9993_CR14 9993_CR36 X Wu (9993_CR46) 2013; 26 9993_CR31 9993_CR10 9993_CR1 FA Fasick (9993_CR12) 1977; 41 9993_CR19 9993_CR3 C Brummer (9993_CR8) 2018; 107 M Kim (9993_CR20) 2017; 44 9993_CR2 9993_CR37 9993_CR16 9993_CR38 9993_CR17 9993_CR18 |
| References_xml | – reference: Zhang J M, Harman M, Ma L (2020) Liu Y, Machine learning testing, Survey, landscapes and horizons. IEEE Trans Softw Eng – reference: Arpteg A, Brinne B, Crnkovic-Friis L, Bosch J (2018) Software engineering challenges of deep learning, IEEE – reference: Martínez-Plumed F, Contreras-Ochando L, Ferri C, Orallo J H, Kull M, Lachiche N, Quintana M J R, Flach PA (2019) CRISP-DM twenty years later: From data mining processes to data science trajectories. IEEE Trans Knowl Data Eng – reference: Akkiraju R, Sinha V, Xu A, Mahmud J, Gundecha P, Liu Z, Liu X, Schumacher J (2020) Characterizing machine learning processes: A maturity framework. In: International conference on business process management, Springer, pp 17–31 – reference: Bernardi L, Mavridis T, Estevez P (2019) 150 successful machine learning models: 6 lessons learned at Booking.com. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. ACM, pp 1743–1751 – reference: LincolnYGubaENaturalistic inquiry1985Newbury ParkSAGE10.1016/0147-1767(85)90062-8 – reference: MeadCIsmailMAnalog VLSI implementation of neural systems1989BerlinSpringer10.1007/978-1-4613-1639-8 – reference: Kavuri AS, Milne A (2019) Fintech and the future of financial services: What are the research gaps CAMA Working Paper No 18/2019 – reference: Ericson G, Rohm WA, Martens J, Sharkey K, Casey C, Harvey B, Schonning N (2017) Team data science process documentation. Retrieved September 2020 https://docs.microsoft.com/en-us/azure/machine-learning/team-data-science-process/ – reference: ING (2019) ING at a glance. https://www.ing.com/About-us/Profile/ING-at-a-glance.htm – reference: Lwakatare LE, Crnkovic I, Rånge E, Bosch J (2020) From a data science driven process to a continuous delivery process for machine learning systems. In: International conference on product-focused software process improvement. Springer, pp 185–201 – reference: Haakman M, Cruz L, Huijgens H, van Deursen A (2020) Machine learning behind the scenes: An exploratory study in fintech - case study protocol. https://1drv.ms/b/s!AuvX_CBP4YARcBBYdYAOG8qLGIk – reference: YinRKCase study research and applications: Design and methods2017Thousand OaksSage publications – reference: MariscalGMarbanOFernandezCA survey of data mining and knowledge discovery process models and methodologiesKnowl Eng Rev201025213716610.1017/S0269888910000032 – reference: Moyle S, Jorge A (2001) Ramsys-a methodology for supporting rapid remote collaborative data mining projects, vol 64 – reference: HalcombEJDavidsonPMIs verbatim transcription of interview data always necessary?Appl Nurs Res2006191384210.1016/j.apnr.2005.06.001 – reference: MaxwellJUnderstanding and validity in qualitative researchHarvard Educ Rev199262327930110.17763/haer.62.3.8323320856251826 – reference: Serban A, van der Blom K, Hoos H, Visser J (2020) Adoption and effects of software engineering best practices in machine learning. In: Empirical software engineering and measurement – reference: Martínez-Plumed F, Contreras-Ochando L, Ferri C, Flach P, Hernández-Orallo J, Kull M, Lachiche N, Ramírez-Quintana MJ (2017) Casp-dm: Context aware standard process for data mining. arXiv:1709.09003 – reference: KimMZimmermannTDeLineRBegelAData scientists in software teams: State of the art and challengesIEEE Trans. Softw. Eng.201744111024103810.1109/TSE.2017.2754374 – reference: ShearerCThe CRISP-DM model: the new blueprint for data miningJ Data Warehous2000541322 – reference: FasickFASome uses of untranscribed tape recordings in survey researchPublic Opinion Q197741454955210.1086/268415 – reference: BrummerCYadavYFintech and the innovation trilemmaGeo LJ2018107235 – reference: LinJRyaboyDScaling big data mining infrastructure: the Twitter experienceAcm SIGKDD Explorations Newsletter201314261910.1145/2481244.2481247 – reference: Nicolescu B, Ertas A (2008) Transdisciplinary theory and practice. USA, TheATLAS – reference: Wu C J, Brooks D, Chen K, Chen D, Choudhury S, Dukhan M, Hazelwood K, Isaac E, Jia Y, Jia B et al (2019) Machine learning at facebook: Understanding inference at the edge, IEEE – reference: 15288 I S O /I E C /I E E E (2015) Systems and software engineering – System life cycle processes. Institute of Electrical and Electronic Engineers (IEEE), New York – reference: Cruz L, Abreu R (2019) Catalog of energy patterns for mobile applications. Empir Softw Eng. https://doi.org/10.1007/s10664-019-09682-0 – reference: WuXZhuXWuGQDingWData mining with big dataIEEE Trans Knowl Data Eng201326197107 – reference: MitchellTRThompsonLPetersonECronkRTemporal adjustments in the evaluation of events: The “rosy view”J Exper Soc Psychol199733442144810.1006/jesp.1997.1333 – reference: Ahmed Z, Amizadeh S, Bilenko M, Carr R, Chin WS, Dekel Y, Dupre X, Eksarevskiy V, Filipi S, Finley T et al (2019) Machine learning at Microsoft with ML. NET. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. pp 2448–2458 – reference: Damiani E, Frati F (2018) Towards conceptual models for machine learning computations. In: International conference on conceptual modeling. Springer, pp 3–9 – reference: Larios Vargas E, Aniche M, Treude C, Bruntink M, Gousios G (2020) Selecting third-party libraries: The practitioners’ perspective. In: Proceedings of the 28th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp 245–256 – reference: Rollins J (2015) Foundational methodology for data science. Domino Data Lab, Inc. Whitepaper – reference: Amershi S, Begel A, Bird C, DeLine R, Gall H, Kamar E, Nagappan N, Nushi B, Zimmermann T (2019) Software engineering for machine learning: a case study. In: Proceedings of the 41st International conference on software engineering, software engineering in practice. IEEE Press, pp 291–300 – reference: Ryan M J, Wheatcraft L S (2017) On the use of the terms verification and validation. In: INCOSE International Symposium, vol 27, pp 1277–1290. https://doi.org/10.1002/j.2334-5837.2017.00427.x – reference: WangYCognitive informatics: A new transdisciplinary research fieldBrain and Mind20034211512710.1023/A:1025419826662https://doi.org/10.1023/A:1025419826662 – reference: Hill C, Bellamy R, Erickson T, Burnett M (2016) Trials and tribulations of developers of intelligent systems: A field study. In: 2016 IEEE symposium on visual languages and human-centric computing (VL/HCC). IEEE, pp 162–170 – reference: MenziesTThe five laws of se for aiIEEE Softw.2019371818510.1109/MS.2019.2954841 – reference: WengrafTQualitative research interviewing: Biographic narrative and semi-structured methods2001Newbury ParkSAGE10.4135/9781849209717 – reference: Stol KJ, Ralph P, Fitzgerald B (2016) Grounded theory in software engineering research: a critical review and guidelines. In: Proceedings of the 38th International conference on software engineering, pp 120–131 – reference: StraussACorbinJBasics of qualitative research1990Thousand OaksSage publications – reference: Breck E, Cai S, Nielsen E, Salib M, Sculley D (2017) The ml test score: A rubric for ml production readiness and technical debt reduction, IEEE – reference: Van LooRMaking innovation more competitive: the case of fintechUCLA L Rev201865232 – reference: Sculley D, Holt G, Golovin D, Davydov E, Phillips T, Ebner D, Chaudhary V, Young M, Crespo JF, Dennison D (2015) Hidden technical debt in machine learning systems. In: Advances in neural information processing systems, pp 2503–2511 – reference: FeredayJMuir-CochraneEDemonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme developmentInt J Qual Methods200651809210.1177/160940690600500107 – reference: Brereton P, Kitchenham B A, Budgen D, Li Z (2008) Using a protocol template for case study planning. In: EASE, Citeseer, vol 8, pp 41–48 – reference: Mitchell M, Wu S, Zaldivar A, Barnes P, Vasserman L, Hutchinson B, Spitzer E, Raji ID, Gebru T (2019) Model cards for model reporting. In: Proceedings of the conference on fairness, accountability, and transparency, pp 220–229 – volume-title: Naturalistic inquiry year: 1985 ident: 9993_CR23 doi: 10.1016/0147-1767(85)90062-8 – ident: 9993_CR24 doi: 10.1007/978-3-030-64148-1_12 – ident: 9993_CR34 – ident: 9993_CR7 doi: 10.14236/ewic/EASE2008.5 – ident: 9993_CR10 doi: 10.1007/978-3-030-00847-5_1 – ident: 9993_CR16 doi: 10.1109/VLHCC.2016.7739680 – ident: 9993_CR11 – ident: 9993_CR40 doi: 10.1145/2884781.2884833 – volume: 5 start-page: 13 issue: 4 year: 2000 ident: 9993_CR39 publication-title: J Data Warehous – ident: 9993_CR6 doi: 10.1109/BigData.2017.8258038 – ident: 9993_CR36 doi: 10.1002/j.2334-5837.2017.00427.x – ident: 9993_CR2 doi: 10.1007/978-3-030-58666-9_2 – volume-title: Analog VLSI implementation of neural systems year: 1989 ident: 9993_CR29 doi: 10.1007/978-1-4613-1639-8 – volume: 65 start-page: 232 year: 2018 ident: 9993_CR42 publication-title: UCLA L Rev – ident: 9993_CR26 – volume: 25 start-page: 137 issue: 2 year: 2010 ident: 9993_CR25 publication-title: Knowl Eng Rev doi: 10.1017/S0269888910000032 – volume: 62 start-page: 279 issue: 3 year: 1992 ident: 9993_CR28 publication-title: Harvard Educ Rev doi: 10.17763/haer.62.3.8323320856251826 – volume-title: Basics of qualitative research year: 1990 ident: 9993_CR41 – ident: 9993_CR14 – ident: 9993_CR18 – ident: 9993_CR37 – volume: 41 start-page: 549 issue: 4 year: 1977 ident: 9993_CR12 publication-title: Public Opinion Q doi: 10.1086/268415 – volume-title: Qualitative research interviewing: Biographic narrative and semi-structured methods year: 2001 ident: 9993_CR44 doi: 10.4135/9781849209717 – ident: 9993_CR4 doi: 10.1109/SEAA.2018.00018 – ident: 9993_CR21 doi: 10.1145/3368089.3409711 – ident: 9993_CR33 – ident: 9993_CR35 – volume: 5 start-page: 80 issue: 1 year: 2006 ident: 9993_CR13 publication-title: Int J Qual Methods doi: 10.1177/160940690600500107 – volume: 44 start-page: 1024 issue: 11 year: 2017 ident: 9993_CR20 publication-title: IEEE Trans. Softw. Eng. doi: 10.1109/TSE.2017.2754374 – volume: 107 start-page: 235 year: 2018 ident: 9993_CR8 publication-title: Geo LJ – ident: 9993_CR9 doi: 10.1007/s10664-019-09682-0 – ident: 9993_CR1 doi: 10.1145/3292500.3330667 – volume: 26 start-page: 97 issue: 1 year: 2013 ident: 9993_CR46 publication-title: IEEE Trans Knowl Data Eng – ident: 9993_CR27 – ident: 9993_CR3 doi: 10.1109/ICSE-SEIP.2019.00042 – volume: 37 start-page: 81 issue: 1 year: 2019 ident: 9993_CR30 publication-title: IEEE Softw. doi: 10.1109/MS.2019.2954841 – volume: 33 start-page: 421 issue: 4 year: 1997 ident: 9993_CR32 publication-title: J Exper Soc Psychol doi: 10.1006/jesp.1997.1333 – volume: 19 start-page: 38 issue: 1 year: 2006 ident: 9993_CR15 publication-title: Appl Nurs Res doi: 10.1016/j.apnr.2005.06.001 – ident: 9993_CR48 doi: 10.1109/TSE.2019.2962027 – volume-title: Case study research and applications: Design and methods year: 2017 ident: 9993_CR47 – ident: 9993_CR31 doi: 10.1145/3287560.3287596 – ident: 9993_CR45 doi: 10.1109/HPCA.2019.00048 – ident: 9993_CR5 doi: 10.1145/3292500.3330744 – ident: 9993_CR19 doi: 10.2139/ssrn.3333515 – ident: 9993_CR17 – volume: 14 start-page: 6 issue: 2 year: 2013 ident: 9993_CR22 publication-title: Acm SIGKDD Explorations Newsletter doi: 10.1145/2481244.2481247 – ident: 9993_CR38 doi: 10.1145/3382494.3410681 – volume: 4 start-page: 115 issue: 2 year: 2003 ident: 9993_CR43 publication-title: Brain and Mind doi: 10.1023/A:1025419826662 |
| SSID | ssj0009745 |
| Score | 2.5744197 |
| Snippet | Tech-leading organizations are embracing the forthcoming artificial intelligence revolution. Intelligent systems are replacing and cooperating with traditional... |
| SourceID | crossref springer |
| SourceType | Enrichment Source Index Database Publisher |
| SubjectTerms | Compilers Computer Science Interpreters Programming Languages Software Engineering/Programming and Operating Systems |
| Subtitle | An exploratory study in Fintech |
| Title | AI lifecycle models need to be revised |
| URI | https://link.springer.com/article/10.1007/s10664-021-09993-1 |
| Volume | 26 |
| WOSCitedRecordID | wos000691590700003&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAVX databaseName: SpringerLink Contemporary Journals customDbUrl: eissn: 1573-7616 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0009745 issn: 1382-3256 databaseCode: RSV dateStart: 19970101 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LS8NAEB6kevBifWJ9sQfxogvJ7mbTHEUseinii97CPqFQUmmi4L93km6sBSnofTaE2f32m2Fn5gM4z1hsFcbVNHNKUWERitqnhuokUYhLjrTgG7GJdDjsj0bZQ2gKK9tq9_ZJsrmpfzS7SSloXVJQRzWcYs6zjnTXr-H4-PS6GLWbNtLE9XA9ypHRQ6vM799YpqPlt9CGYgbd__3cNmyFkJJcz8_ADqy5Yhe6rVwDCejdg4vrezIZe2c-0Yw0EjglKZC9SDUl2pG64Ld0dh9eBrfPN3c0yCRQw7K4ol5bo1lkmedC61R5KxIhUo-hl42dYFFqmZGYx7nIWJV6h_ugM0zjvEC4ZoofQKeYFu4QCK7RklvpMYwTRvNMam5cxKxxiTRG9CBuvZWbMEO8lrKY5Ivpx7UjcnRE3jgij3tw-b3mbT5BY6X1VevgPKCpXGF-9DfzY9hk8z2iUXwCnWr27k5hw3xU43J21hyjL_crvyM |
| linkProvider | Springer Nature |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LS8NAEB5EBb1Yn1ifexAvupDsbjbmWMTSYi2iVXoL2RcUSipNFPz3TtLEWpCC3mdDmN1vvxl2Zj6Ai4j5JsG4mkY2SagwCEXlQk1VECSIS4604EqxibDfvxkOo8eqKSyrq93rJ8nypv7R7CaloEVJQRHVcIo5z5pAxioK-Z6eX-ejdsNSmrgYrkc5MnrVKvP7NxbpaPEttKSYduN_P7cNW1VISVqzM7ADKzbdhUYt10Aq9O7BZatLxiNn9SeakVICJyMpshfJJ0RZUhT8Ztbsw0v7bnDboZVMAtUs8nPqlNGKeYY5LpQKE2dEIEToMPQyvhXMCw3TEvM462mThM7iPqgI0zgnEK5Rwg9gNZ2k9hAIrlGSG-kwjBNa8Ugqrq3HjLaB1Fo0wa-9FetqhnghZTGO59OPC0fE6Ii4dETsN-Hqe83bbILGUuvr2sFxhaZsifnR38zPYaMzeOjFvW7__hg22Wy_qOefwGo-fbensK4_8lE2PSuP1Bf3bcIH |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LSwMxEA5SRbxYn1ifOYgXDd1Nshv3WNRiUUrBB70tmxcUyrZ0V8F_72QftgUpiPdJCJN8zAyZ-T6ELiPq6wTyahKZJCFcAxSlFYrIIEgAlwzCgi3EJkS_fzscRoOFKf6i273-kixnGhxLU5q3p9q2FwbfwpAT117gMhxGoP5Z5040yNXrL-9z2l1RyBQ7oj3CILpXYzO_77Ecmpb_RYtw023-_6A7aLtKNXGnfBu7aM2ke6hZyzjgCtX76KrTw-ORNeoLzHAhjZPhFKIazidYGuwagTOjD9Bb9-H17pFU8glE0cjPiZVaSeppahmXUiRW84BzYSEl077h1BOaqhDqO-MpnQhr4H5kBOWd5QDjKGGHqJFOUnOEMKyRIdOhhfSOK8miUDJlPKqVCUKleAv5tediVXGLO4mLcTxnRXaOiMERceGI2G-h658105JZY6X1Te3suEJZtsL8-G_mF2hzcN-Nn3v9pxO0RcvrIp5_ihr57MOcoQ31mY-y2Xnxur4Bfz_K6w |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=AI+lifecycle+models+need+to+be+revised&rft.jtitle=Empirical+software+engineering+%3A+an+international+journal&rft.au=Haakman%2C+Mark&rft.au=Cruz%2C+Lu%C3%ADs&rft.au=Huijgens%2C+Hennie&rft.au=van+Deursen%2C+Arie&rft.date=2021-09-01&rft.pub=Springer+US&rft.issn=1382-3256&rft.eissn=1573-7616&rft.volume=26&rft.issue=5&rft_id=info:doi/10.1007%2Fs10664-021-09993-1&rft.externalDocID=10_1007_s10664_021_09993_1 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1382-3256&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1382-3256&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1382-3256&client=summon |