Detection and evaluation of bias-inducing features in machine learning

The cause-to-effect analysis can help us decompose all the likely causes of a problem, such as an undesirable business situation or unintended harm to the individual(s). This implies that we can identify how the problems are inherited, rank the causes to help prioritize fixes, simplify a complex pro...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	Empirical software engineering : an international journal Ročník 29; číslo 1; s. 22
Hlavní autori:	Openja, Moses, Laberge, Gabriel, Khomh, Foutse
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	New York Springer US 01.02.2024 Springer Nature B.V Springer Verlag
Predmet:	Bias Compilers Computer Science Context Decision making Evaluation Interpreters Machine learning Programming Languages Software Engineering/Programming and Operating Systems Special Issue on Equitable Data and Technology Subject specialists Bias Sensitive features Fairness Machine learning
ISSN:	1382-3256, 1573-7616
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	The cause-to-effect analysis can help us decompose all the likely causes of a problem, such as an undesirable business situation or unintended harm to the individual(s). This implies that we can identify how the problems are inherited, rank the causes to help prioritize fixes, simplify a complex problem and visualize them. In the context of machine learning (ML), one can use cause-to-effect analysis to understand the reason for the biased behavior of the system. For example, we can examine the root causes of biases by checking each feature for a potential cause of bias in the model. To approach this, one can apply small changes to a given feature or a pair of features in the data, following some guidelines and observing how it impacts the decision made by the model (i.e., model prediction). Therefore, we can use cause-to-effect analysis to identify the potential bias-inducing features, even when these features are originally are unknown. This is important since most current methods require a pre-identification of sensitive features for bias assessment and can actually miss other relevant bias-inducing features, which is why systematic identification of such features is necessary. Moreover, it often occurs that to achieve an equitable outcome, one has to take into account sensitive features in the model decision. Therefore, it should be up to the domain experts to decide based on their knowledge of the context of a decision whether bias induced by specific features is acceptable or not. In this study, we propose an approach for systematically identifying all bias-inducing features of a model to help support the decision-making of domain experts. Our technique is based on the idea of swapping the values of the features and computing the divergences in the distribution of the model prediction using different distance functions. We evaluated our technique using four well-known datasets to showcase how our contribution can help spearhead the standard procedure when developing, testing, maintaining, and deploying fair/equitable machine learning systems.
AbstractList	The cause-to-effect analysis can help us decompose all the likely causes of a problem, such as an undesirable business situation or unintended harm to the individual(s). This implies that we can identify how the problems are inherited, rank the causes to help prioritize fixes, simplify a complex problem and visualize them. In the context of machine learning (ML), one can use cause-to-effect analysis to understand the reason for the biased behavior of the system. For example, we can examine the root causes of biases by checking each feature for a potential cause of bias in the model. To approach this, one can apply small changes to a given feature or a pair of features in the data, following some guidelines and observing how it impacts the decision made by the model (i.e., model prediction). Therefore, we can use cause-to-effect analysis to identify the potential bias-inducing features, even when these features are originally are unknown. This is important since most current methods require a pre-identification of sensitive features for bias assessment and can actually miss other relevant bias-inducing features, which is why systematic identification of such features is necessary. Moreover, it often occurs that to achieve an equitable outcome, one has to take into account sensitive features in the model decision. Therefore, it should be up to the domain experts to decide based on their knowledge of the context of a decision whether bias induced by specific features is acceptable or not. In this study, we propose an approach for systematically identifying all bias-inducing features of a model to help support the decision-making of domain experts. Our technique is based on the idea of swapping the values of the features and computing the divergences in the distribution of the model prediction using different distance functions. We evaluated our technique using four well-known datasets to showcase how our contribution can help spearhead the standard procedure when developing, testing, maintaining, and deploying fair/equitable machine learning systems. The cause-to-effect analysis can help us decompose all the likely causes of a problem, such as an undesirable business situation or unintended harm to the individual(s). This implies that we can identify how the problems are inherited, rank the causes to help prioritize fixes, simplify a complex problem and visualize them. In the context of machine learning (ML), one can use cause-to-effect analysis to understand the reason for the biased behavior of the system. For example, we can examine the root causes of biases by checking each feature for a potential cause of bias in the model. To approach this, one can apply small changes to a given feature or a pair of features in the data, following some guidelines and observing how it impacts the decision made by the model (i.e., model prediction). Therefore, we can use cause-to-effect analysis to identify the potential bias-inducing features, even when these features are originally are unknown. This is important since most current methods require a pre-identification of sensitive features for bias assessment and can actually miss other relevant bias-inducing features, which is why systematic identification of such features is necessary. Moreover, it often occurs that to achieve an equitable outcome, one has to take into account sensitive features in the model decision. Therefore, it should be up to the domain experts to decide based on their knowledge of the context of a decision whether bias induced by specific features is acceptable or not. In this study, we propose an
ArticleNumber	22
Author	Khomh, Foutse Openja, Moses Laberge, Gabriel
Author_xml	– sequence: 1 givenname: Moses orcidid: 0000-0002-4115-3834 surname: Openja fullname: Openja, Moses email: openja.moses@polymtl.ca organization: Polytechnique Montreal – sequence: 2 givenname: Gabriel surname: Laberge fullname: Laberge, Gabriel organization: Polytechnique Montreal – sequence: 3 givenname: Foutse surname: Khomh fullname: Khomh, Foutse organization: Polytechnique Montreal
BackLink	https://hal.science/hal-04265505$$DView record in HAL
BookMark	eNp9kUtLxDAUhYMo-PwDrgquXERv3u1SfMOAG12HNE2cDDXVpBX892amiuBiFiG5N9-5HO45RLtxiA6hUwIXBEBdZgJScgyUYQIcGix20AERimElidwtb1ZTzKiQ--gw5xUANIqLA3R340ZnxzDEysSucp-mn8ymHHzVBpNxiN1kQ3ytvDPjlFyuQqzejF2G6KremRTL5zHa86bP7uTnPkIvd7fP1w948XT_eH21wJbVZMROtL5rOOUeLFMt77pWegmqhY6C8paQpua1NeBEbS3toPVGMUt903BCjWRH6HyeuzS9fk_hzaQvPZigH64Wet0DTqUQID5JYc9m9j0NH5PLo14NU4rFnqYNkaIsi6rtFJByGr6m6pmyacg5Oa9tGDd7GpMJvSag1znoOQddRutNDloUKf0n_fW9VcRmUS5wfHXpz9UW1TdZ85qq
CitedBy_id	crossref_primary_10_1145_3766890 crossref_primary_10_1007_s11831_024_10134_2 crossref_primary_10_1145_3680463 crossref_primary_10_1016_j_jechem_2024_10_032
Cites_doi	10.1145/3097983.3098095 10.1145/2939672.2939778 10.1145/3510003.3510091 10.1093/bioinformatics/btr300 10.1109/ICSME55016.2022.00014 10.1145/3368089.3409697 10.1145/1754428.1754432 10.1007/s10506-014-9156-4 10.1145/3468264.3468537 10.1037/h0037350 10.1287/ited.2013.0124 10.1086/713744 10.1016/0004-3702(86)90090-1 10.1007/s10664-022-10116-7 10.1089/big.2016.0047 10.1145/3530019.3530039 10.1109/ICDM.2012.45 10.1145/3377811.3380377 10.24251/HICSS.2018.668 10.1109/SANER.2018.8330212 10.1016/j.dss.2014.03.001 10.1007/s10618-010-0190-x 10.1109/TKDE.2012.72 10.1145/1568234.1568252 10.1145/3377811.3380331 10.1146/annurev.psych.58.110405.085542 10.1109/CICYBS.2011.5949405 10.1257/jep.34.2.68 10.1145/2020408.2020488 10.1007/978-94-010-9521-1_5 10.1371/journal.pone.0108358 10.1007/s10115-011-0463-8 10.1097/00001648-199203000-00013 10.1007/s10115-006-0040-8 10.1109/ICSME55016.2022.00010 10.1080/10618600.2014.907095 10.1017/S0269888913000039 10.1016/j.chb.2019.04.019 10.1145/3290605.3300773 10.1016/j.spl.2011.04.001 10.1080/01621459.1986.10478354 10.3390/app11146271 10.1093/ije/dyt127 10.1016/0002-9149(89)90524-9 10.1017/CBO9780511614491.005 10.1002/9781119945710 10.1109/ISIT.2004.1365067 10.1145/3338906.3338937
ContentType	Journal Article
Copyright	Crown 2023 Crown 2023. Attribution - NonCommercial
Copyright_xml	– notice: Crown 2023 – notice: Crown 2023. – notice: Attribution - NonCommercial
DBID	AAYXX CITATION 7SC 8FD 8FE 8FG ABJCF AFKRA ARAPS BENPR BGLVJ CCPQU DWQXO HCIFZ JQ2 L6V L7M L~C L~D M7S P5Z P62 PHGZM PHGZT PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PTHSS S0W 1XC VOOES
DOI	10.1007/s10664-023-10409-5
DatabaseName	CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central UK/Ireland Advanced Technologies & Computer Science Collection ProQuest Central Technology collection ProQuest One ProQuest Central SciTech Premium Collection ProQuest Computer Science Collection ProQuest Engineering Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional Engineering Database Advanced Technologies & Aerospace Database ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Premium ProQuest One Academic ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection DELNET Engineering & Technology Collection Hyper Article en Ligne (HAL) Hyper Article en Ligne (HAL) (Open Access)
DatabaseTitle	CrossRef Technology Collection Technology Research Database Computer and Information Systems Abstracts – Academic ProQuest One Academic Middle East (New) ProQuest Advanced Technologies & Aerospace Collection ProQuest Computer Science Collection Computer and Information Systems Abstracts SciTech Premium Collection ProQuest One Community College ProQuest Central China ProQuest Central ProQuest One Applied & Life Sciences ProQuest Engineering Collection ProQuest Central Korea ProQuest Central (New) Advanced Technologies Database with Aerospace Engineering Collection Advanced Technologies & Aerospace Collection Engineering Database ProQuest One Academic Eastern Edition ProQuest Technology Collection ProQuest SciTech Collection Computer and Information Systems Abstracts Professional Advanced Technologies & Aerospace Database ProQuest One Academic UKI Edition ProQuest DELNET Engineering and Technology Collection Materials Science & Engineering Collection ProQuest One Academic ProQuest One Academic (New)
DatabaseTitleList	Technology Collection Technology Collection
Database_xml	– sequence: 1 dbid: BENPR name: ProQuest Central url: https://www.proquest.com/central sourceTypes: Aggregation Database
DeliveryMethod	fulltext_linktorsrc
Discipline	Computer Science
EISSN	1573-7616
ExternalDocumentID	oai:HAL:hal-04265505v1 10_1007_s10664_023_10409_5
GrantInformation_xml	– fundername: Natural Sciences and Engineering Research Council of Canada grantid: RGPIN-2019-06956 funderid: http://dx.doi.org/10.13039/501100000038
GroupedDBID	-4Z -59 -5G -BR -EM -Y2 -~C .86 .DC .VR 06D 0R~ 0VY 199 1N0 1SB 2.D 203 28- 29G 2J2 2JN 2JY 2KG 2LR 2P1 2VQ 2~H 30V 4.4 406 408 409 40D 40E 5GY 5QI 5VS 67Z 6NX 78A 8FE 8FG 8TC 8UJ 95- 95. 95~ 96X AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AAOBN AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYOK AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDZT ABECU ABFTD ABFTV ABHLI ABHQN ABJCF ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTEG ABTHY ABTKH ABTMW ABULA ABWNU ABXPI ACAOD ACBXY ACDTI ACGFS ACHSB ACHXU ACIWK ACKNC ACMDZ ACMLO ACOKC ACOMO ACPIV ACSNA ACZOJ ADHHG ADHIR ADIMF ADINQ ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF ADZKW AEBTG AEFIE AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFEXP AFGCZ AFKRA AFLOW AFQWF AFWTZ AFZKB AGAYW AGDGC AGGDS AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMXSW AMYLF AMYQR AOCGG ARAPS ARMRJ ASPBG AVWKF AXYYD AYJHY AZFZN B-. BA0 BBWZM BDATZ BENPR BGLVJ BGNMA BSONS CAG CCPQU COF CS3 CSCUP DDRTE DL5 DNIVK DPUIP DU5 EBLON EBS EIOEI EJD ESBYG FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNWQR GQ6 GQ7 GQ8 GXS H13 HCIFZ HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ I09 IHE IJ- IKXTQ ITM IWAJR IXC IZIGR IZQ I~X I~Z J-C J0Z JBSCW JCJTX JZLTJ KDC KOV KOW L6V LAK LLZTM M4Y M7S MA- N2Q NB0 NDZJH NPVJJ NQJWS NU0 O9- O93 O9G O9I O9J OAM P19 P62 P9O PF0 PT4 PT5 PTHSS Q2X QOK QOS R4E R89 R9I RHV RNI RNS ROL RPX RSV RZC RZE RZK S0W S16 S1Z S26 S27 S28 S3B SAP SCJ SCLPG SCO SDH SDM SHX SISQX SJYHP SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 T16 TSG TSK TSV TUC U2A UG4 UOJIU UTJUX UZXMN VC2 VFIZW W23 W48 WK8 YLTOR Z45 Z7R Z7S Z7V Z7X Z7Z Z81 Z83 Z86 Z88 Z8M Z8N Z8P Z8R Z8T Z8U Z8W Z92 ZMTXR ~EX AAPKM AAYXX ABBRH ABDBE ABFSG ABRTQ ACSTC ADHKG AEZWR AFDZB AFFHD AFHIU AFOHR AGQPQ AHPBZ AHWEU AIXLP ATHPR AYFIA CITATION PHGZM PHGZT PQGLB 7SC 8FD DWQXO JQ2 L7M L~C L~D PKEHL PQEST PQQKQ PQUKI PRINS 1XC VOOES
ID	FETCH-LOGICAL-c381t-e5bfd9424f0c37b4ddb6f607b0d207fc119848ca0e58cc2d0bfa73c2f99412a63
IEDL.DBID	M7S
ISICitedReferencesCount	2
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001124077200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1382-3256
IngestDate	Sat Nov 29 15:00:27 EST 2025 Tue Dec 02 16:29:00 EST 2025 Tue Dec 02 16:05:06 EST 2025 Tue Nov 18 21:53:06 EST 2025 Sat Nov 29 05:37:47 EST 2025 Fri Feb 21 02:44:19 EST 2025
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	1
Keywords	Bias Sensitive features Fairness Machine learning
Language	English
License	Attribution - NonCommercial: http://creativecommons.org/licenses/by-nc
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c381t-e5bfd9424f0c37b4ddb6f607b0d207fc119848ca0e58cc2d0bfa73c2f99412a63
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-4115-3834
OpenAccessLink	https://hal.science/hal-04265505
PQID	2901290947
PQPubID	326341
ParticipantIDs	hal_primary_oai_HAL_hal_04265505v1 proquest_journals_2916502327 proquest_journals_2901290947 crossref_citationtrail_10_1007_s10664_023_10409_5 crossref_primary_10_1007_s10664_023_10409_5 springer_journals_10_1007_s10664_023_10409_5
PublicationCentury	2000
PublicationDate	2024-02-01
PublicationDateYYYYMMDD	2024-02-01
PublicationDate_xml	– month: 02 year: 2024 text: 2024-02-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York – name: Dordrecht
PublicationSubtitle	An International Journal
PublicationTitle	Empirical software engineering : an international journal
PublicationTitleAbbrev	Empir Software Eng
PublicationYear	2024
Publisher	Springer US Springer Nature B.V Springer Verlag
Publisher_xml	– name: Springer US – name: Springer Nature B.V – name: Springer Verlag
References	Janos A, Steinbrunn W, Pfisterer M, Detrano R (1998) Heart disease data set. https://archive.ics.uci.edu/ml/datasets/heart+disease BlankRMDabadyMCitroCFBlankRMMeasuring racial discrimination2004DCNational Academies Press Washington Fisher V (2016) University of Texas at austin Buolamwini J, Gebru T (2018) Gender shades: intersectional accuracy disparities in commercial gender classification. In: Conference on fairness, accountability and transparency, pp 77–91. PMLR de Kleer J, Brown JS (1986) Theories of causal ordering. Artif Intell 29(1):33–61. https://doi.org/10.1016/0004-3702(86)90090-1. www.sciencedirect.com/science/article/pii/0004370286900901 ChouldechovaAFair prediction with disparate impact: a study of bias in recidivism prediction instrumentsBig Data20175215316310.1089/big.2016.0047 RuggieriSPedreschiDTuriniFData mining for discrimination discoveryACM Trans Knowl Discov Data (TKDD)20104214010.1145/1754428.1754432 MancuhanKCliftonCCombating discrimination using bayesian networksArtif Intell Law201422221123810.1007/s10506-014-9156-4 DetranoRJanosiASteinbrunnWPfistererMSchmidJJSandhuSGuppyKHLeeSFroelicherVInternational application of a new probability algorithm for the diagnosis of coronary artery diseaseAm J Cardiol198964530431010.1016/0002-9149(89)90524-9 FryeCRowatCFeigeIAsymmetric shapley values: incorporating causal knowledge into model-agnostic explainabilityAdv Neural Inf Process Syst20203312291239 Arcidiacono P (2018b) Professor Peter Arcidiacono provides expert analysis for nonprofit’s lawsuit against harvard. https://econ.duke.edu/news/professor-peter-arcidiacono-provides-expert-analysis-nonprofit%E2%80%99s-lawsuit-against-harvard Fuglede B, Topsoe F (2004) Jensen-shannon divergence and hilbert space embedding. In: International symposium oninformation theory, 2004. ISIT 2004. Proceedings, pp 31. https://doi.org/10.1109/ISIT.2004.1365067 KalousisAPradosJHilarioMStability of feature selection algorithms: a study on high-dimensional spacesKnowl Inf Syst20071219511610.1007/s10115-006-0040-8 LoohuisLOCaravagnaGGraudenziARamazzottiDMauriGAntoniottiMMishraBInferring tree causal models of cancer progression with probability raisingPloS One201491010.1371/journal.pone.0108358 PhelpsESThe statistical theory of racism and sexismAm Econ Rev1972624659661 Robins JM, Greenland S (1992) Identifiability and exchangeability for direct and indirect effects. Epidemiology 143–155 Openja M, Nikanjam A, Yahmed AH, Khomh F, Jiang ZMJ (2022b) An empirical study of challenges in converting deep learning models. In: 2022 IEEE International conference on software maintenance and evolution (ICSME), IEEE, pp 13–23 PearlJMackenzieDThe book of why: the new science of cause and effect2018Basic Books De CapitaniLDe MartiniDOn stochastic orderings of the wilcoxon rank sum test statistic—with applications to reproducibility probability estimation testingStat Probab Lett2011818937946280372810.1016/j.spl.2011.04.001 HollandPWCausation and raceETS Res Rep Ser200320031i21 ToloşiLLengauerTClassification with correlated features: unreliability of feature ranking and solutionsBioinformatics201127141986199410.1093/bioinformatics/btr300 Liu Y, Li Y, Guo J, Zhou Y, Xu B (2018) Connecting software metrics across versions to predict defects. In: 2018 IEEE 25th International conference on software analysis, evolution and reengineering (SANER), IEEE, pp 232–243 Ribeiro MT, Singh S, Guestrin C (2016) Why should i trust you? explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1135–1144 Majumder S, Chakraborty J, Bai GR, Stolee KT, Menzies T (2021) Fair enough: searching for sufficient measures of fairness. arXiv:2110.13029 Agarwal A, Beygelzimer A, Dudík M, Langford J, Wallach H (2018) A reductions approach to fair classification. In: International conference on machine learning, pp 60–69. PMLR BerzuiniCDawidPBernardinellLCausality: statistical perspectives and applications2012John Wiley & Sons10.1002/9781119945710 RomeiARuggieriSA multidisciplinary survey on discrimination analysisKnowl Eng Rev201429558263810.1017/S0269888913000039 Chakraborty J, Majumder S, Yu, Z, Menzies T (2020) Fairway: a way to build fair ml software. In: Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pp 654–665 PereraAAletiATantithamthavornCJiarpakdeeJTurhanBKuhnLWalkerKSearch-based fairness testing for regression-based machine learning systemsEmpir Softw Eng202227313610.1007/s10664-022-10116-7 Pearl J (2001) Direct and indirect effects. In: Proceedings of the seventeenth conference on uncertainty in artificial intelligence, UAI’01, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, pp 411–420 Pearl J et al (2000) Models, reasoning and inference. Cambridge, UK: Cambridge UniversityPress 19(2) Willenborg L, De Waal T (2012) Elements of statistical disclosure control, vol 155. Springer Science & Business Media Corbett-Davies S, Pierson E, Feller A, Goel S, Huq A (2017) Algorithmic decision making and the cost of fairness. In: Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining, pp 797–806 Zhang P, Wang J, Sun J, Dong G, Wang X, Wang X, Dong JS, Dai T (2020) White-box fairness testing through adversarial sampling. In: Proceedings of the ACM/IEEE 42nd international conference on software engineering, pp 949–960 Kamiran F, Karim A, Zhang X (2012) Decision theory for discrimination-aware classification. In: 2012 IEEE 12th International conference on data mining, IEEE, pp 924–929 MacKinnonDPFairchildAJFritzMSMediation analysisAnnu Rev Psychol20075859310.1146/annurev.psych.58.110405.085542 Kuczmarski J (2018) Reducing gender bias in google translate. Google Blog 6 Peng K, Chakraborty J, Menzies T (2021) Fairmask: better fairness via model-based rebalancing of protected attributes. arXiv:2110.01109 Romano J, Kromrey JD, Coraggio J, Skowronek J, Devine L (2006) Exploring methods for evaluating group differences on the nsse and other surveys: are the t-test and cohen’sd indices the most appropriate choices. In: Annual meeting of the southern association for institutional research, Citeseer, pp 1–51 LangKKahn-Lang SpitzerARace discrimination: an economic perspectiveJ Econ Perspect2020342688910.1257/jep.34.2.68 CaldersTVerwerSThree naive bayes approaches for discrimination-free classificationData Min Knowl Disc2010212277292272050710.1007/s10618-010-0190-x Freedman DA (2005) On specifying graphical models for causation, and the identification problem. Identif Infer Econ Model pp 56–79 Bhattacharya A (2022) Applied machine learning explainability techniques: make ML models explainable and trustworthy for practical applications using LIME, SHAP, and more. Packt Publishing Ltd Openja M, Majidi F, Khomh F, Chembakottu B, Li H (2022a) Studying the practices of deploying machine learning projects on docker. In: Proceedings of the 26th international conference on evaluation and assessment in software engineering, pp 190–200 RichiardiLBelloccoRZugnaDMediation analysis in epidemiology: methods, interpretation and biasInt J Epidemiol20134251511151910.1093/ije/dyt127 ShinDParkYJRole of fairness, accountability, and transparency in algorithmic affordanceComput Hum Behav20199827728410.1016/j.chb.2019.04.019 Arrow K (1971) The theory of discrimination. Working Papers 403, Princeton University, Department of Economics, industrial relations section. https://EconPapers.repec.org/RePEc:pri:indrel:30a Li Y, Meng L, Chen L, Yu L, Wu D, Zhou Y, Xu B (2022) Training data debugging for the fairness of machine learning software, pp 2215–2227. Association for Computing Machinery, New York, USA. https://doi.org/10.1145/3510003.3510091 Barbosa NM, Chen M (2019) Rehumanized crowdsourcing: a labeling framework addressing bias and ethics in machine learning. In: Proceedings of the 2019 CHI conference on human factors in computing systems, pp 1–12 TofallisCAdd or multiply? a tutorial on ranking and choosing with multiple criteriaINFORMS Trans Educ201414310911910.1287/ited.2013.0124 SunsteinCRLegal reasoning and political conflict2018Oxford University Press FisherARudinCDominiciFAll models are wrong, but many are useful: learning a variable’s importance by studying an entire class of prediction models simultaneouslyJ Mach Learn Res2019201771813911408 Hajian S, Domingo-Ferrer J, Martinez-Balleste A (2011) Discrimination prevention in data mining for intrusion and crime detection. In: 2011 IEEE symposium on computational intelligence in cyber security (CICS), IEEE, pp 47–54 Arcidiacono P (2018a) Expert report of Peter S. Arcidiacono students for fair admissions, inc. v. harvard no. 14-cv-14176-adb (d. mass) KamiranFCaldersTData preprocessing techniques for classification without discriminationKnowl Inf Syst201233113310.1007/s10115-011-0463-8 Yapo A, Weiss J (2018) Ethical implications of bias in machine learning BarocasSHardtMNarayananAFairness in machine learningNips Tutorial201712 Majidi F, Openja M, Khomh F, Li H (2022) An empirical study on the usage of automated machine learning tools. In: 2022 IEEE International conference on software maintenance and evolution (ICSME), IEEE , pp 59–70 WachterSMittelstadtBRussellCCounterfactual explanations without opening the black box: automated decisions and the gdprHarv JL & Tech201731841 Pleiss G, Raghavan M, Wu F, Kleinberg J, Weinberger KQ (2017) On fairness and calibration. Advances Neural Inf Process Syst 30 Chakraborty J, Majumder S, Menzies T (2021) Bias in machine learning software: why? how? what to do? In: Proceedings of the 29th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pp 429–440 GoldsteinAKapelnerABleichJPitkinEPeeking inside the black box: visualizing statistical learning with plots of individual conditional expectationJ Comput Graph Stat20152414465332824710.1080/10618600.2014.907095 Luong BT, Rugg 10409_CR72 S Barocas (10409_CR10) 2017; 1 10409_CR32 CR Sunstein (10409_CR73) 2018 10409_CR74 T Calders (10409_CR15) 2010; 21 A Romei (10409_CR68) 2014; 29 C Frye (10409_CR28) 2020; 33 DP MacKinnon (10409_CR49) 2007; 58 L De Capitani (10409_CR22) 2011; 81 S Alelyani (10409_CR4) 2021; 11 R Detrano (10409_CR24) 1989; 64 10409_CR79 10409_CR33 10409_CR78 D Aha (10409_CR3) 1988; 3 10409_CR37 10409_CR38 10409_CR60 A Perera (10409_CR61) 2022; 27 LO Loohuis (10409_CR46) 2014; 9 10409_CR20 10409_CR64 10409_CR21 S Moro (10409_CR53) 2014; 62 10409_CR63 P Arcidiacono (10409_CR7) 2022; 40 D Shin (10409_CR71) 2019; 98 S Wachter (10409_CR77) 2017; 31 A Chouldechova (10409_CR18) 2017; 5 K Mancuhan (10409_CR52) 2014; 22 DB Rubin (10409_CR69) 1974; 66 10409_CR25 10409_CR66 10409_CR23 10409_CR67 K Lang (10409_CR43) 2020; 34 10409_CR29 10409_CR27 L Toloşi (10409_CR76) 2011; 27 10409_CR50 10409_CR54 10409_CR51 A Goldstein (10409_CR30) 2015; 24 S Hajian (10409_CR31) 2012; 25 10409_CR19 C Tofallis (10409_CR75) 2014; 14 C Hitchcock (10409_CR34) 2012 10409_CR57 10409_CR14 10409_CR55 10409_CR12 10409_CR56 10409_CR17 10409_CR59 10409_CR16 A Fisher (10409_CR26) 2019; 20 F Kamiran (10409_CR40) 2012; 33 10409_CR80 10409_CR9 10409_CR42 J Pearl (10409_CR58) 2018 10409_CR8 10409_CR41 ES Phelps (10409_CR62) 1972; 62 10409_CR5 10409_CR6 10409_CR1 C Berzuini (10409_CR11) 2012 10409_CR2 RM Blank (10409_CR13) 2004 PW Holland (10409_CR36) 2003; 2003 PW Holland (10409_CR35) 1986; 81 L Richiardi (10409_CR65) 2013; 42 10409_CR47 10409_CR44 10409_CR45 A Kalousis (10409_CR39) 2007; 12 10409_CR48 S Ruggieri (10409_CR70) 2010; 4
References_xml	– reference: Ribeiro MT, Singh S, Guestrin C (2016) Why should i trust you? explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1135–1144 – reference: Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions. Adv Neural Inf Process Syst 30 – reference: Kuczmarski J (2018) Reducing gender bias in google translate. Google Blog 6 – reference: DetranoRJanosiASteinbrunnWPfistererMSchmidJJSandhuSGuppyKHLeeSFroelicherVInternational application of a new probability algorithm for the diagnosis of coronary artery diseaseAm J Cardiol198964530431010.1016/0002-9149(89)90524-9 – reference: HollandPWStatistics and causal inferenceJ Am Stat Assoc19868139694596086761810.1080/01621459.1986.10478354 – reference: Majidi F, Openja M, Khomh F, Li H (2022) An empirical study on the usage of automated machine learning tools. In: 2022 IEEE International conference on software maintenance and evolution (ICSME), IEEE , pp 59–70 – reference: SunsteinCRLegal reasoning and political conflict2018Oxford University Press – reference: De CapitaniLDe MartiniDOn stochastic orderings of the wilcoxon rank sum test statistic—with applications to reproducibility probability estimation testingStat Probab Lett2011818937946280372810.1016/j.spl.2011.04.001 – reference: PereraAAletiATantithamthavornCJiarpakdeeJTurhanBKuhnLWalkerKSearch-based fairness testing for regression-based machine learning systemsEmpir Softw Eng202227313610.1007/s10664-022-10116-7 – reference: Peng K, Chakraborty J, Menzies T (2021) Fairmask: better fairness via model-based rebalancing of protected attributes. arXiv:2110.01109 – reference: RomeiARuggieriSA multidisciplinary survey on discrimination analysisKnowl Eng Rev201429558263810.1017/S0269888913000039 – reference: TofallisCAdd or multiply? a tutorial on ranking and choosing with multiple criteriaINFORMS Trans Educ201414310911910.1287/ited.2013.0124 – reference: Hardt M, Price E, Srebro N (2016) Equality of opportunity in supervised learning. Adv. Neural Inf Process Syst 29 – reference: MacKinnonDPFairchildAJFritzMSMediation analysisAnnu Rev Psychol20075859310.1146/annurev.psych.58.110405.085542 – reference: MancuhanKCliftonCCombating discrimination using bayesian networksArtif Intell Law201422221123810.1007/s10506-014-9156-4 – reference: AlelyaniSDetection and evaluation of machine learning biasAppl Sci20211114627110.3390/app11146271 – reference: PearlJMackenzieDThe book of why: the new science of cause and effect2018Basic Books – reference: Luong BT, Ruggieri S, Turini F (2011) k-nn as an implementation of situation testing for discrimination discovery and prevention. In: Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining, pp 502–510 – reference: RuggieriSPedreschiDTuriniFData mining for discrimination discoveryACM Trans Knowl Discov Data (TKDD)20104214010.1145/1754428.1754432 – reference: Pearl J (2001) Direct and indirect effects. In: Proceedings of the seventeenth conference on uncertainty in artificial intelligence, UAI’01, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, pp 411–420 – reference: Liu Y, Li Y, Guo J, Zhou Y, Xu B (2018) Connecting software metrics across versions to predict defects. In: 2018 IEEE 25th International conference on software analysis, evolution and reengineering (SANER), IEEE, pp 232–243 – reference: de Kleer J, Brown JS (1986) Theories of causal ordering. Artif Intell 29(1):33–61. https://doi.org/10.1016/0004-3702(86)90090-1. www.sciencedirect.com/science/article/pii/0004370286900901 – reference: Hajian S, Domingo-Ferrer J, Martinez-Balleste A (2011) Discrimination prevention in data mining for intrusion and crime detection. In: 2011 IEEE symposium on computational intelligence in cyber security (CICS), IEEE, pp 47–54 – reference: Corbett-Davies S, Pierson E, Feller A, Goel S, Huq A (2017) Algorithmic decision making and the cost of fairness. In: Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining, pp 797–806 – reference: BlankRMDabadyMCitroCFBlankRMMeasuring racial discrimination2004DCNational Academies Press Washington – reference: HollandPWCausation and raceETS Res Rep Ser200320031i21 – reference: Romano J, Kromrey JD, Coraggio J, Skowronek J, Devine L (2006) Exploring methods for evaluating group differences on the nsse and other surveys: are the t-test and cohen’sd indices the most appropriate choices. In: Annual meeting of the southern association for institutional research, Citeseer, pp 1–51 – reference: Chakraborty J, Majumder S, Menzies T (2021) Bias in machine learning software: why? how? what to do? In: Proceedings of the 29th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pp 429–440 – reference: KamiranFCaldersTData preprocessing techniques for classification without discriminationKnowl Inf Syst201233113310.1007/s10115-011-0463-8 – reference: HitchcockCZaltaENProbabilistic causationThe Stanford Encyclopedia of Philosophy, Winter20122012Metaphysics Research LabStanford University – reference: Kamiran F, Karim A, Zhang X (2012) Decision theory for discrimination-aware classification. In: 2012 IEEE 12th International conference on data mining, IEEE, pp 924–929 – reference: Johnson B, Brun Y, Meliou A (2020) Causal testing: understanding defects’ root causes. In: Proceedings of the ACM/IEEE 42nd international conference on software engineering, pp 87–99 – reference: Agarwal A, Beygelzimer A, Dudík M, Langford J, Wallach H (2018) A reductions approach to fair classification. In: International conference on machine learning, pp 60–69. PMLR – reference: Arcidiacono P (2018b) Professor Peter Arcidiacono provides expert analysis for nonprofit’s lawsuit against harvard. https://econ.duke.edu/news/professor-peter-arcidiacono-provides-expert-analysis-nonprofit%E2%80%99s-lawsuit-against-harvard – reference: FisherARudinCDominiciFAll models are wrong, but many are useful: learning a variable’s importance by studying an entire class of prediction models simultaneouslyJ Mach Learn Res2019201771813911408 – reference: RichiardiLBelloccoRZugnaDMediation analysis in epidemiology: methods, interpretation and biasInt J Epidemiol20134251511151910.1093/ije/dyt127 – reference: BarocasSHardtMNarayananAFairness in machine learningNips Tutorial201712 – reference: Suppes P (1970) A theory of probabilistic causality – reference: Buolamwini J, Gebru T (2018) Gender shades: intersectional accuracy disparities in commercial gender classification. In: Conference on fairness, accountability and transparency, pp 77–91. PMLR – reference: Bhattacharya A (2022) Applied machine learning explainability techniques: make ML models explainable and trustworthy for practical applications using LIME, SHAP, and more. Packt Publishing Ltd – reference: KalousisAPradosJHilarioMStability of feature selection algorithms: a study on high-dimensional spacesKnowl Inf Syst20071219511610.1007/s10115-006-0040-8 – reference: Pedreschi D, Ruggieri S, Turini F (2009) Integrating induction and deduction for finding evidence of discrimination. In: Proceedings of the 12th international conference on artificial intelligence and law, ICAIL ’09, Association for Computing Machinery, New York, USA, pp 157–166. https://doi.org/10.1145/1568234.1568252 – reference: FryeCRowatCFeigeIAsymmetric shapley values: incorporating causal knowledge into model-agnostic explainabilityAdv Neural Inf Process Syst20203312291239 – reference: Openja M, Nikanjam A, Yahmed AH, Khomh F, Jiang ZMJ (2022b) An empirical study of challenges in converting deep learning models. In: 2022 IEEE International conference on software maintenance and evolution (ICSME), IEEE, pp 13–23 – reference: Willenborg L, De Waal T (2012) Elements of statistical disclosure control, vol 155. Springer Science & Business Media – reference: Aggarwal A, Lohia P, Nagar S, Dey K, Saha D (2019) Black box fairness testing of machine learning models. In: Proceedings of the 2019 27th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp 625–635 – reference: RubinDBEstimating causal effects of treatments in randomized and nonrandomized studiesJ Educ Psychol197466568810.1037/h0037350 – reference: Robins JM, Greenland S (1992) Identifiability and exchangeability for direct and indirect effects. Epidemiology 143–155 – reference: Fuglede B, Topsoe F (2004) Jensen-shannon divergence and hilbert space embedding. In: International symposium oninformation theory, 2004. ISIT 2004. Proceedings, pp 31. https://doi.org/10.1109/ISIT.2004.1365067 – reference: Freedman DA (2005) On specifying graphical models for causation, and the identification problem. Identif Infer Econ Model pp 56–79 – reference: HajianSDomingo-FerrerJA methodology for direct and indirect discrimination prevention in data miningIEEE Trans Knowl Data Eng20122571445145910.1109/TKDE.2012.72 – reference: ShinDParkYJRole of fairness, accountability, and transparency in algorithmic affordanceComput Hum Behav20199827728410.1016/j.chb.2019.04.019 – reference: Openja M, Majidi F, Khomh F, Chembakottu B, Li H (2022a) Studying the practices of deploying machine learning projects on docker. In: Proceedings of the 26th international conference on evaluation and assessment in software engineering, pp 190–200 – reference: Arcidiacono P (2018a) Expert report of Peter S. Arcidiacono students for fair admissions, inc. v. harvard no. 14-cv-14176-adb (d. mass) – reference: Fisher V (2016) University of Texas at austin – reference: Yapo A, Weiss J (2018) Ethical implications of bias in machine learning – reference: Custers B, Calders T, Schermer B, Zarsky T (1866) Discrimination and privacy in the information society. Stud Appl Philos Epistemology Rational Ethics 3 – reference: ChouldechovaAFair prediction with disparate impact: a study of bias in recidivism prediction instrumentsBig Data20175215316310.1089/big.2016.0047 – reference: Janos A, Steinbrunn W, Pfisterer M, Detrano R (1998) Heart disease data set. https://archive.ics.uci.edu/ml/datasets/heart+disease – reference: Majumder S, Chakraborty J, Bai GR, Stolee KT, Menzies T (2021) Fair enough: searching for sufficient measures of fairness. arXiv:2110.13029 – reference: Chakraborty J, Majumder S, Yu, Z, Menzies T (2020) Fairway: a way to build fair ml software. In: Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pp 654–665 – reference: Pleiss G, Raghavan M, Wu F, Kleinberg J, Weinberger KQ (2017) On fairness and calibration. Advances Neural Inf Process Syst 30 – reference: LangKKahn-Lang SpitzerARace discrimination: an economic perspectiveJ Econ Perspect2020342688910.1257/jep.34.2.68 – reference: BerzuiniCDawidPBernardinellLCausality: statistical perspectives and applications2012John Wiley & Sons10.1002/9781119945710 – reference: Li Y, Meng L, Chen L, Yu L, Wu D, Zhou Y, Xu B (2022) Training data debugging for the fairness of machine learning software, pp 2215–2227. Association for Computing Machinery, New York, USA. https://doi.org/10.1145/3510003.3510091 – reference: Simon HA (1977) Causal ordering and identifiability. Models of Discovery: and other topics in the methods of science pp 53–80 – reference: ToloşiLLengauerTClassification with correlated features: unreliability of feature ranking and solutionsBioinformatics201127141986199410.1093/bioinformatics/btr300 – reference: Barbosa NM, Chen M (2019) Rehumanized crowdsourcing: a labeling framework addressing bias and ethics in machine learning. In: Proceedings of the 2019 CHI conference on human factors in computing systems, pp 1–12 – reference: Zhang P, Wang J, Sun J, Dong G, Wang X, Wang X, Dong JS, Dai T (2020) White-box fairness testing through adversarial sampling. In: Proceedings of the ACM/IEEE 42nd international conference on software engineering, pp 949–960 – reference: Arrow K (1971) The theory of discrimination. Working Papers 403, Princeton University, Department of Economics, industrial relations section. https://EconPapers.repec.org/RePEc:pri:indrel:30a – reference: AhaDKiblerDInstance-based prediction of heart-disease presence with the cleveland databaseUniv Calif19883132 – reference: PhelpsESThe statistical theory of racism and sexismAm Econ Rev1972624659661 – reference: LoohuisLOCaravagnaGGraudenziARamazzottiDMauriGAntoniottiMMishraBInferring tree causal models of cancer progression with probability raisingPloS One201491010.1371/journal.pone.0108358 – reference: ArcidiaconoPKinslerJRansomTLegacy and athlete preferences at harvardJ Labor Econ202240113315610.1086/713744 – reference: GoldsteinAKapelnerABleichJPitkinEPeeking inside the black box: visualizing statistical learning with plots of individual conditional expectationJ Comput Graph Stat20152414465332824710.1080/10618600.2014.907095 – reference: Corbett-Davies S, Goel S (2018) The measure and mismeasure of fairness: a critical review of fair machine learning. arXiv preprint arXiv:1808.00023 – reference: MoroSCortezPRitaPA data-driven approach to predict the success of bank telemarketingDecis Support Syst201462223110.1016/j.dss.2014.03.001 – reference: CaldersTVerwerSThree naive bayes approaches for discrimination-free classificationData Min Knowl Disc2010212277292272050710.1007/s10618-010-0190-x – reference: Pearl J et al (2000) Models, reasoning and inference. Cambridge, UK: Cambridge UniversityPress 19(2) – reference: WachterSMittelstadtBRussellCCounterfactual explanations without opening the black box: automated decisions and the gdprHarv JL & Tech201731841 – ident: 10409_CR57 – ident: 10409_CR20 doi: 10.1145/3097983.3098095 – ident: 10409_CR64 doi: 10.1145/2939672.2939778 – ident: 10409_CR44 doi: 10.1145/3510003.3510091 – volume: 27 start-page: 1986 issue: 14 year: 2011 ident: 10409_CR76 publication-title: Bioinformatics doi: 10.1093/bioinformatics/btr300 – ident: 10409_CR50 doi: 10.1109/ICSME55016.2022.00014 – ident: 10409_CR63 – ident: 10409_CR17 doi: 10.1145/3368089.3409697 – volume: 4 start-page: 1 issue: 2 year: 2010 ident: 10409_CR70 publication-title: ACM Trans Knowl Discov Data (TKDD) doi: 10.1145/1754428.1754432 – ident: 10409_CR47 – volume: 62 start-page: 659 issue: 4 year: 1972 ident: 10409_CR62 publication-title: Am Econ Rev – ident: 10409_CR5 – ident: 10409_CR14 – volume: 22 start-page: 211 issue: 2 year: 2014 ident: 10409_CR52 publication-title: Artif Intell Law doi: 10.1007/s10506-014-9156-4 – volume: 3 start-page: 3 issue: 1 year: 1988 ident: 10409_CR3 publication-title: Univ Calif – ident: 10409_CR16 doi: 10.1145/3468264.3468537 – ident: 10409_CR1 – volume: 66 start-page: 688 issue: 5 year: 1974 ident: 10409_CR69 publication-title: J Educ Psychol doi: 10.1037/h0037350 – volume: 14 start-page: 109 issue: 3 year: 2014 ident: 10409_CR75 publication-title: INFORMS Trans Educ doi: 10.1287/ited.2013.0124 – ident: 10409_CR33 – volume-title: Legal reasoning and political conflict year: 2018 ident: 10409_CR73 – ident: 10409_CR12 – ident: 10409_CR6 – volume: 40 start-page: 133 issue: 1 year: 2022 ident: 10409_CR7 publication-title: J Labor Econ doi: 10.1086/713744 – ident: 10409_CR23 doi: 10.1016/0004-3702(86)90090-1 – ident: 10409_CR37 – volume: 27 start-page: 1 issue: 3 year: 2022 ident: 10409_CR61 publication-title: Empir Softw Eng doi: 10.1007/s10664-022-10116-7 – volume: 5 start-page: 153 issue: 2 year: 2017 ident: 10409_CR18 publication-title: Big Data doi: 10.1089/big.2016.0047 – ident: 10409_CR54 doi: 10.1145/3530019.3530039 – ident: 10409_CR41 doi: 10.1109/ICDM.2012.45 – volume: 33 start-page: 1229 year: 2020 ident: 10409_CR28 publication-title: Adv Neural Inf Process Syst – ident: 10409_CR38 doi: 10.1145/3377811.3380377 – volume-title: Measuring racial discrimination year: 2004 ident: 10409_CR13 – ident: 10409_CR79 doi: 10.24251/HICSS.2018.668 – ident: 10409_CR45 doi: 10.1109/SANER.2018.8330212 – ident: 10409_CR51 – volume: 62 start-page: 22 year: 2014 ident: 10409_CR53 publication-title: Decis Support Syst doi: 10.1016/j.dss.2014.03.001 – volume: 21 start-page: 277 issue: 2 year: 2010 ident: 10409_CR15 publication-title: Data Min Knowl Disc doi: 10.1007/s10618-010-0190-x – volume: 25 start-page: 1445 issue: 7 year: 2012 ident: 10409_CR31 publication-title: IEEE Trans Knowl Data Eng doi: 10.1109/TKDE.2012.72 – ident: 10409_CR59 doi: 10.1145/1568234.1568252 – ident: 10409_CR80 doi: 10.1145/3377811.3380331 – volume: 58 start-page: 593 year: 2007 ident: 10409_CR49 publication-title: Annu Rev Psychol doi: 10.1146/annurev.psych.58.110405.085542 – ident: 10409_CR32 doi: 10.1109/CICYBS.2011.5949405 – volume: 34 start-page: 68 issue: 2 year: 2020 ident: 10409_CR43 publication-title: J Econ Perspect doi: 10.1257/jep.34.2.68 – volume: 1 start-page: 2 year: 2017 ident: 10409_CR10 publication-title: Nips Tutorial – ident: 10409_CR48 doi: 10.1145/2020408.2020488 – ident: 10409_CR72 doi: 10.1007/978-94-010-9521-1_5 – ident: 10409_CR78 – volume: 9 issue: 10 year: 2014 ident: 10409_CR46 publication-title: PloS One doi: 10.1371/journal.pone.0108358 – volume: 33 start-page: 1 issue: 1 year: 2012 ident: 10409_CR40 publication-title: Knowl Inf Syst doi: 10.1007/s10115-011-0463-8 – volume-title: The book of why: the new science of cause and effect year: 2018 ident: 10409_CR58 – ident: 10409_CR66 doi: 10.1097/00001648-199203000-00013 – volume: 12 start-page: 95 issue: 1 year: 2007 ident: 10409_CR39 publication-title: Knowl Inf Syst doi: 10.1007/s10115-006-0040-8 – volume: 20 start-page: 1 issue: 177 year: 2019 ident: 10409_CR26 publication-title: J Mach Learn Res – volume: 31 start-page: 841 year: 2017 ident: 10409_CR77 publication-title: Harv JL & Tech – ident: 10409_CR55 doi: 10.1109/ICSME55016.2022.00010 – volume: 24 start-page: 44 issue: 1 year: 2015 ident: 10409_CR30 publication-title: J Comput Graph Stat doi: 10.1080/10618600.2014.907095 – volume: 29 start-page: 582 issue: 5 year: 2014 ident: 10409_CR68 publication-title: Knowl Eng Rev doi: 10.1017/S0269888913000039 – volume: 98 start-page: 277 year: 2019 ident: 10409_CR71 publication-title: Comput Hum Behav doi: 10.1016/j.chb.2019.04.019 – ident: 10409_CR9 doi: 10.1145/3290605.3300773 – ident: 10409_CR8 – volume: 81 start-page: 937 issue: 8 year: 2011 ident: 10409_CR22 publication-title: Stat Probab Lett doi: 10.1016/j.spl.2011.04.001 – ident: 10409_CR56 – ident: 10409_CR60 – ident: 10409_CR21 – volume-title: The Stanford Encyclopedia of Philosophy, Winter year: 2012 ident: 10409_CR34 – volume: 81 start-page: 945 issue: 396 year: 1986 ident: 10409_CR35 publication-title: J Am Stat Assoc doi: 10.1080/01621459.1986.10478354 – ident: 10409_CR67 – ident: 10409_CR25 – volume: 2003 start-page: i issue: 1 year: 2003 ident: 10409_CR36 publication-title: ETS Res Rep Ser – ident: 10409_CR74 – volume: 11 start-page: 6271 issue: 14 year: 2021 ident: 10409_CR4 publication-title: Appl Sci doi: 10.3390/app11146271 – ident: 10409_CR42 – ident: 10409_CR19 – volume: 42 start-page: 1511 issue: 5 year: 2013 ident: 10409_CR65 publication-title: Int J Epidemiol doi: 10.1093/ije/dyt127 – volume: 64 start-page: 304 issue: 5 year: 1989 ident: 10409_CR24 publication-title: Am J Cardiol doi: 10.1016/0002-9149(89)90524-9 – ident: 10409_CR27 doi: 10.1017/CBO9780511614491.005 – volume-title: Causality: statistical perspectives and applications year: 2012 ident: 10409_CR11 doi: 10.1002/9781119945710 – ident: 10409_CR29 doi: 10.1109/ISIT.2004.1365067 – ident: 10409_CR2 doi: 10.1145/3338906.3338937
SSID	ssj0009745
Score	2.3874633
Snippet	The cause-to-effect analysis can help us decompose all the likely causes of a problem, such as an undesirable business situation or unintended harm to the...
SourceID	hal proquest crossref springer
SourceType	Open Access Repository Aggregation Database Enrichment Source Index Database Publisher
StartPage	22
SubjectTerms	Bias Compilers Computer Science Context Decision making Evaluation Interpreters Machine learning Programming Languages Software Engineering/Programming and Operating Systems Special Issue on Equitable Data and Technology Subject specialists
SummonAdditionalLinks	– databaseName: SpringerLINK Contemporary 1997-Present dbid: RSV link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LT8MwDI7Y4MCF8RSDgSLEDSIl6fs4AdMOaEK8tFvVvGASdGjd9vtxsnYbiCHBoZc6aSM7iW3Z_ozQeRhmkjIJlpunbJhRBUQwJYnFWWGwqWPDlWs2EfV6cb-f3JVFYUWV7V6FJN1NvVTsFoY-AR0DVwd4JSSooXVQd7E9jvcPzwuo3ci1JrbgesQDjV6Wyvz8jS_qqPZqkyGXLM1vwVGnczqN_612G22VNiZuzzbFDlrT-S5qVP0bcHmc91DnWo9dJlaOs1zhBfA3HhosBllBwGGfSPgtNtoBgBZ4kON3l3-pcdlw4mUfPXVuHq-6pOyrQCTo5zHRgTAq8blvqPQi4StbihfSSFDFaWQkY0nsxzKjOoil5IoKk0We5CZJfMaz0DtA9XyY60OEE6n90HiRlwFNKBCu0FQKboNvCaeiiVjF3lSWoOO298VbuoBLtoxKgVGpY1QaNNHFfM7HDHLj19FnILX5QIuW3W3fpvaddQ-tAzZlTdSqhJqWZ7RIbQQZnsSPVpAZWK9gcAL5spLxgrx6RUd_G36MNjkYSrNM8Baqj0cTfYI25HQ8KEanbmt_AhGE7zg priority: 102 providerName: Springer Nature
Title	Detection and evaluation of bias-inducing features in machine learning
URI	https://link.springer.com/article/10.1007/s10664-023-10409-5 https://www.proquest.com/docview/2901290947 https://www.proquest.com/docview/2916502327 https://hal.science/hal-04265505
Volume	29
WOSCitedRecordID	wos001124077200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVAVX databaseName: SpringerLINK Contemporary 1997-Present customDbUrl: eissn: 1573-7616 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0009745 issn: 1382-3256 databaseCode: RSV dateStart: 19970101 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LT9wwEB7xOvQCpaXqUkBW1Ru1cJyHk1NFKSsO1WrFo0JcovjVrlSyQBZ-f2e8DksrwaWH-BA7keWxPTOe8fcBfCqKxojEoOWWWgoz2pzrxBpOOCsJTurSSxvIJtRoVF5eVuN44NbFtMp-TwwbtZ0aOiM_kGjH5KhgpPpyc8uJNYqiq5FCYxlWCSVBhtS9swXorgokxQSzx1PU7fHSTLw6VxQZxx_iRoQ-Ds__UkzLvygt8onN-U-YNGif4cb_9vs1rEe7kx3OJ8omLLn2DWz0nA4sLvG3MPzmZiE7q2VNa9kCDJxNPdOTpuPoxN8b7CTzLoCCdmzSsuuQk-lYJKH4uQUXw-PzoxMeuRa4QZ094y7X3laZzLwwqdKZpet5hVBaWCmUN0lSlVlpGuHy0hhphfaNSo30VZUlsinSd7DSTlv3HlhlXFb4VKUN1mmLAtdOGC0pIFdJoQeQ9ANdmwhETnwYv-sFhDIJp8ZRq4Nw6nwA-4_f3MxhOF5s_RHl99iQELRPDr_X9I5cRnLKHpIB7PQCq-O67WqKKuNTZeqZ6l6YA_jcz4hF9fM92n75bx_glURjaZ4NvgMrs7t7twtr5mE26e72YPXr8Wh8uhcmN5bj_ArL07MffwB1Avzy
linkProvider	ProQuest
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Lb9QwEB61BQkulKdYKGAhOIFVx3HizQGhirLaqssKiSL1ZuIXrATZ0myL-FP8Rma8SReQ2lsPHHKJnSi2P49nMjPfADwry9qJzKHmlntyM_qC28w7TjwrGYJ6GKVPxSb0dDo8PKzer8GvPheGwip7mZgEtZ87-ke-LVGPKfCAkfr10XdOVaPIu9qX0FjCYj_8_IEmW_tqbxfX97mUo7cHb8a8qyrAHZ5OCx4KG32lpIrC5doqT4lopdBWeCl0dBma4WroahGKoXPSCxtrnTsZq0plsi5zfO86XFEk_VOo4IcVya9ORZGJ1o_nqEt0STpdql5ZKo4DQMGHNhUv_joI179QGOYfOu4_btl02o02_7d5ugk3Or2a7Sw3wi1YC81t2OxrVrBOhN2B0W5YpOizhtWNZyuyczaPzM7qls8af-JwUlgMifS0ZbOGfUsxp4F1RTY-34WPlzKae7DRzJtwH1jlgipjrvMa26xHQNsgnJXkcKyksAPI-oU1riNap3ofX82KIprAYHCVTAKDKQbw4uyZoyXNyIW9nyJezjoSQ_h4Z2LoHpnEZHSeZgPY6gFiOrnUGvKa41UpfU5zD54BvOwRuGo-_4seXPy2J3BtfPBuYiZ70_2HcF2iYriMfN-CjcXxSXgEV93pYtYeP04bisGny0bmbwkcV3o
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Lb9QwEB61BSEulKe6UMBCcAILx3l4c0CoYlm1arXaA0gVFxO_YKWSLc22iL_Gr2PG63QBqb31wCGX2Ili-_N4JjPzDcDzqmqsyCxqbrkjN6Mrucmc5cSzkiGoh0G6WGxCTSbDw8N6uga_-lwYCqvsZWIU1G5u6R_5a4l6TIkHDJrqIYVFTEfjt8ffOVWQIk9rX05jCZF9__MHmm_dm70RrvULKcfvP7zb5anCALd4Ui24L01wdSGLIGyuTOEoKa0SyggnhQo2Q5O8GNpG-HJorXTChEblVoa6LjLZVDm-dx2uKbQxyfCblp9WhL8qFkgmij-eo16REnZS2l5VFRwHg0IQ7Ste_nUorn-lkMw_9N1_XLTx5Btv_s9zdhtuJX2b7Sw3yB1Y8-1d2OxrWbAk2u7BeOQXMSqtZU3r2IoEnc0DM7Om47PWnVqcIBZ8JEPt2Kxl32Isqmep-MaX-_DxSkbzADbaeeu3gNXWF1XIVd5gm3EIdOOFNZIckbUUZgBZv8jaJgJ2qgNypFfU0QQMjSumIzB0OYCX588cL-lHLu39DLFz3pGYw3d3DjTdI1OZjNGzbADbPVh0kledJm86XnWhLmjugTSAVz0aV80Xf9HDy9_2FG4gIPXB3mT_EdyUqC8uA-K3YWNxcuofw3V7tph1J0_i3mLw-aqB-RtrhGBd
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Detection+and+evaluation+of+bias-inducing+features+in+machine+learning&rft.jtitle=Empirical+software+engineering+%3A+an+international+journal&rft.au=Openja%2C+Moses&rft.au=Laberge%2C+Gabriel&rft.au=Khomh%2C+Foutse&rft.date=2024-02-01&rft.pub=Springer+Nature+B.V&rft.issn=1382-3256&rft.eissn=1573-7616&rft.volume=29&rft.issue=1&rft.spage=22&rft_id=info:doi/10.1007%2Fs10664-023-10409-5&rft.externalDBID=HAS_PDF_LINK
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1382-3256&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1382-3256&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1382-3256&client=summon