Astraea: Grammar-Based Fairness Testing

Software often produces biased outputs. In particular, machine learning (ML) based software is known to produce erroneous predictions when processing discriminatory inputs . Such unfair program behavior can be caused by societal bias. In the last few years, Amazon, Microsoft and Google have provided...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE transactions on software engineering Ročník 48; číslo 12; s. 5188 - 5211
Hlavní autori:	Soremekun, Ezekiel, Udeshi, Sakshi, Chattopadhyay, Sudipta
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	New York IEEE 01.12.2022 IEEE Computer Society
Predmet:	Bias Fault diagnosis Grammar Grammars Machine learning Natural language processing program debugging Sentiment analysis Software software fairness Software systems Software testing Task analysis Test pattern generators Testing Violations
ISSN:	0098-5589, 1939-3520
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	Software often produces biased outputs. In particular, machine learning (ML) based software is known to produce erroneous predictions when processing discriminatory inputs . Such unfair program behavior can be caused by societal bias. In the last few years, Amazon, Microsoft and Google have provided software services that produce unfair outputs, mostly due to societal bias (e.g., gender or race). In such events, developers are saddled with the task of conducting fairness testing . Fairness testing is challenging; developers are tasked with generating discriminatory inputs that reveal and explain biases . We propose a grammar-based fairness testing approach (called Astraea ) which leverages context-free grammars to generate discriminatory inputs that reveal fairness violations in software systems. Using probabilistic grammars, Astraea also provides fault diagnosis by isolating the cause of observed software bias. Astraea 's diagnoses facilitate the improvement of ML fairness. Astraea was evaluated on 18 software systems that provide three major natural language processing (NLP) services. In our evaluation, Astraea generated fairness violations at a rate of about 18%. Astraea generated over 573K discriminatory test cases and found over 102K fairness violations. Furthermore, Astraea improves software fairness by about 76% via model-retraining, on average.
AbstractList	Software often produces biased outputs. In particular, machine learning (ML) based software is known to produce erroneous predictions when processing discriminatory inputs . Such unfair program behavior can be caused by societal bias. In the last few years, Amazon, Microsoft and Google have provided software services that produce unfair outputs, mostly due to societal bias (e.g., gender or race). In such events, developers are saddled with the task of conducting fairness testing . Fairness testing is challenging; developers are tasked with generating discriminatory inputs that reveal and explain biases . We propose a grammar-based fairness testing approach (called Astraea ) which leverages context-free grammars to generate discriminatory inputs that reveal fairness violations in software systems. Using probabilistic grammars, Astraea also provides fault diagnosis by isolating the cause of observed software bias. Astraea ’s diagnoses facilitate the improvement of ML fairness. Astraea was evaluated on 18 software systems that provide three major natural language processing (NLP) services. In our evaluation, Astraea generated fairness violations at a rate of about 18%. Astraea generated over 573K discriminatory test cases and found over 102K fairness violations. Furthermore, Astraea improves software fairness by about 76% via model-retraining, on average.
Author	Soremekun, Ezekiel Udeshi, Sakshi Chattopadhyay, Sudipta
Author_xml	– sequence: 1 givenname: Ezekiel orcidid: 0000-0002-0039-8106 surname: Soremekun fullname: Soremekun, Ezekiel email: ezekiel.soremekun@uni.lu organization: Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Esch-sur-Alzette, Luxembourg – sequence: 2 givenname: Sakshi orcidid: 0000-0001-6484-6249 surname: Udeshi fullname: Udeshi, Sakshi email: sakshi_udeshi@mymail.sutd.edu.sg organization: Singapore University of Technology and Design, Singapore – sequence: 3 givenname: Sudipta orcidid: 0000-0002-4843-5391 surname: Chattopadhyay fullname: Chattopadhyay, Sudipta email: sudipta_chattopadhyay@sutd.edu.sg organization: Singapore University of Technology and Design, Singapore
BookMark	eNp9kL9PQjEUhRuDiYDuJi4kDk4Pb29ff7khATQhcRDnprzXmhJ4D9vH4H9vCcTBweku57sn5xuQXtM2jpBbCmNKQT-u3mdjBMQxoyWVXF2QPtVMF4wj9EgfQKuCc6WvyCClDQBwKXmfPExSF62zT6NFtLudjcWzTa4ezW2IjUtptHKpC83nNbn0dpvczfkOycd8tpq-FMu3xet0siwq1LQral0DSrROA2oUqi59pcqaAgNhqQKhpUflRW0lVkCVX5fWrT2uueaV0siG5P70dx_br0PuNpv2EJtcaVCWigsmmcgpcUpVsU0pOm-q0NkutE0eE7aGgjlKMVmKOUoxZykZhD_gPoa8-vs_5O6EBOfcb1wLqYBK9gOjZWwQ
CODEN	IESEDJ
CitedBy_id	crossref_primary_10_1007_s10664_023_10402_y crossref_primary_10_1109_ACCESS_2024_3488904 crossref_primary_10_1145_3672455 crossref_primary_10_1016_j_jss_2024_112090 crossref_primary_10_1109_TSE_2021_3136169 crossref_primary_10_1145_3652155 crossref_primary_10_1109_ACCESS_2024_3407753 crossref_primary_10_1109_ACCESS_2025_3574462
Cites_doi	10.18653/v1/2020.acl-main.442 10.18653/v1/W18-2501 10.18653/v1/P19-1161 10.1145/3062341.3062349 10.1145/3368089.3409754 10.1109/ICDM.2010.50 10.18653/v1/P19-1159 10.1145/2090236.2090255 10.1109/CVPR.2015.7298640 10.1145/3468264.3468565 10.1109/ICSE.2019.00108 10.1109/TSE.2019.2953066 10.1177/0011392118816743 10.1093/bjps/axi147 10.1145/3368089.3409704 10.1109/QRS.2015.30 10.1609/aaai.v34i05.6267 10.1145/3324884.3416609 10.1145/3194770.3194776 10.18653/v1/N18-2003 10.18653/v1/P16-2096 10.18653/v1/2021.acl-long.149 10.1017/S0269888921000102 10.1109/TSE.2014.2372785 10.1109/TSE.2021.3136169 10.1145/945645.945658 10.1145/2939672.2939778 10.1145/3377811.3380331 10.1162/089120101753342653 10.1145/3468264.3468536 10.1145/3468264.3468537 10.18653/v1/D18-1521 10.1145/3377811.3380420 10.18653/v1/S18-2005 10.1609/aaai.v35i15.17594 10.1145/3133904 10.18653/v1/W19-3811 10.1007/s10603-005-3307-7 10.1145/230538.230561 10.1145/3173574.3173986 10.1080/01621459.1974.10482962 10.24963/ijcai.2020/64 10.18653/v1/2020.acl-main.485 10.18653/v1/D17-1323 10.18653/v1/D19-1531 10.1126/science.aal4230 10.1145/3278721.3278779 10.1145/3238147.3238165 10.1109/ASE.2019.00027 10.2307/2291267 10.18653/v1/D16-1120 10.1109/ICDMW.2009.83 10.1609/aaai.v32i1.11491 10.1145/3338906.3338937 10.1145/1985793.1985795 10.1145/3468264.3473117 10.1145/3368089.3409697 10.1109/TSE.2019.2962027 10.1145/3106237.3106277 10.18653/v1/N18-2002
ContentType	Journal Article
Copyright	Copyright IEEE Computer Society 2022
Copyright_xml	– notice: Copyright IEEE Computer Society 2022
DBID	97E RIA RIE AAYXX CITATION JQ2 K9.
DOI	10.1109/TSE.2022.3141758
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef ProQuest Computer Science Collection ProQuest Health & Medical Complete (Alumni)
DatabaseTitle	CrossRef ProQuest Health & Medical Complete (Alumni) ProQuest Computer Science Collection
DatabaseTitleList	ProQuest Health & Medical Complete (Alumni)
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Computer Science
EISSN	1939-3520
EndPage	5211
ExternalDocumentID	10_1109_TSE_2022_3141758 9678017
Genre	orig-research
GrantInformation_xml	– fundername: Singapore Ministry of Education – fundername: MOE grantid: MOE2018-T2-1-098 – fundername: University of Luxembourg, Ezekiel Soremekun – fundername: University of Luxembourg grantid: AUDACITY-2019-Laiwyers – fundername: OneConnect Financial grantid: RGOCFT2001
GroupedDBID	--Z -DZ -~X .DC 0R~ 29I 4.4 5GY 6IK 85S 8R4 8R5 97E AAJGR AARMG AASAJ AAWTH ABAZT ABPPZ ABQJQ ABVLG ACGFO ACGOD ACIWK ACNCT AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BKOMP BPEOZ CS3 DU5 EBS EDO EJD HZ~ I-F IEDLZ IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P Q2X RIA RIE RNS RXW S10 TAE TN5 TWZ UHB UPT WH7 YZZ .4S 3EH 5VS 7WY 7X7 88E 88I 8FE 8FG 8FI 8FJ 8FL 8G5 9M8 AAYXX ABFSI ABJCF ABUWG ADBBV AETIX AFFHD AFKRA AGSQL AI. AIBXA ALLEH ARAPS ARCSS AZQEC BENPR BEZIV BGLVJ BPHCQ BVXVI CCPQU CITATION DWQXO E.L FRNLG FYUFA GNUQQ GROUPED_ABI_INFORM_RESEARCH GUQSH HCIFZ HMCUK H~9 IBMZZ ICLAB IFJZH ITG ITH K60 K6V K6~ K7- L6V M0C M1P M1Q M2O M2P M7S OHT P62 PHGZM PHGZT PJZUB PPXIY PQBIZ PQBZA PQGLB PQQKQ PROAC PSQYO PTHSS RNI RZB UKHRP UQL VH1 XOL YYP ZCG JQ2 K9.
ID	FETCH-LOGICAL-c291t-d9d0272ae9029268d4fc84d10306a180697f28f6da72c018fb4aebf2b595c8923
IEDL.DBID	RIE
ISICitedReferencesCount	17
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000896667600025&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0098-5589
IngestDate	Sun Nov 30 03:49:00 EST 2025 Tue Nov 18 22:45:01 EST 2025 Sat Nov 29 03:10:27 EST 2025 Wed Aug 27 02:14:29 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	12
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c291t-d9d0272ae9029268d4fc84d10306a180697f28f6da72c018fb4aebf2b595c8923
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0001-6484-6249 0000-0002-4843-5391 0000-0002-0039-8106
PQID	2748563736
PQPubID	21418
PageCount	24
ParticipantIDs	crossref_primary_10_1109_TSE_2022_3141758 ieee_primary_9678017 proquest_journals_2748563736 crossref_citationtrail_10_1109_TSE_2022_3141758
PublicationCentury	2000
PublicationDate	2022-12-01
PublicationDateYYYYMMDD	2022-12-01
PublicationDate_xml	– month: 12 year: 2022 text: 2022-12-01 day: 01
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on software engineering
PublicationTitleAbbrev	TSE
PublicationYear	2022
Publisher	IEEE IEEE Computer Society
Publisher_xml	– name: IEEE – name: IEEE Computer Society
References	ref57 ref13 ref56 ref12 ref14 ref53 (ref66) 2020 ref55 ref54 ref10 (ref2) 2020 wolf (ref76) 2019 ref17 ref16 blodgett (ref18) 2017 dola (ref31) 2021 ref50 ref45 gonen (ref37) 2019 ref48 ref86 ref42 ref85 ref41 ref88 ref44 charmaz (ref24) 2006 blodgett (ref15) 2020 ref49 (ref1) 2021 ref8 ref7 ref9 ref4 ref3 devlin (ref29) 2019 ref5 zafar (ref78) 2017 ref82 ref81 ref40 ref84 ref83 ref80 ref35 ref34 (ref51) 2020 (ref58) 2020 ref36 ref74 ref30 berend (ref11) 2020 ref77 ref33 ref32 ref39 ref38 johnson (ref47) 2020 zemel (ref79) 2013 denton (ref26) 2019 lundberg (ref52) 2017 ref70 ref73 ref72 radford (ref59) 2019; 1 bolukbasi (ref19) 2016 ref68 tian (ref71) 2020 ref23 crawford (ref25) 2017 ref69 ref64 ref20 ref63 (ref43) 2018 ref22 ref65 solaiman (ref67) 2019 ref27 caliskan (ref21) 2017; 356 (ref75) 2020 angwin (ref6) 2016 žliobaite (ref87) 2011 (ref46) 2020 ref60 ref62 ref61 dev (ref28) 2019
References_xml	– ident: ref63 doi: 10.18653/v1/2020.acl-main.442 – ident: ref36 doi: 10.18653/v1/W18-2501 – ident: ref88 doi: 10.18653/v1/P19-1161 – ident: ref10 doi: 10.1145/3062341.3062349 – ident: ref40 doi: 10.1145/3368089.3409754 – ident: ref48 doi: 10.1109/ICDM.2010.50 – ident: ref69 doi: 10.18653/v1/P19-1159 – year: 2019 ident: ref76 article-title: State-of-the-art neural coreference resolution for chatbots – ident: ref32 doi: 10.1145/2090236.2090255 – start-page: 4349 year: 2016 ident: ref19 article-title: Man is to computer programmer as woman is to homemaker? debiasing word embeddings publication-title: Proc Adv Neural Inf Process Syst 29 Annu Conf Neural Inf Process Syst – start-page: 609 year: 2019 ident: ref37 article-title: Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them publication-title: Proc Conf North Amer Chapter Assoc Comput Linguistics Hum Lang Technol – ident: ref57 doi: 10.1109/CVPR.2015.7298640 – ident: ref44 doi: 10.1145/3468264.3468565 – ident: ref49 doi: 10.1109/ICSE.2019.00108 – year: 2020 ident: ref2 – ident: ref73 doi: 10.1109/TSE.2019.2953066 – ident: ref12 doi: 10.1177/0011392118816743 – year: 2020 ident: ref46 – year: 2019 ident: ref67 article-title: Release strategies and the social impacts of language models – ident: ref38 doi: 10.1093/bjps/axi147 – start-page: 879 year: 2019 ident: ref28 article-title: Attenuating bias in word vectors publication-title: Proc 22nd Int Conf Artif Intell Statist – ident: ref13 doi: 10.1145/3368089.3409704 – year: 2020 ident: ref47 article-title: Fairkit, fairkit, on the wall, who's the fairest of them all? supporting data scientists in training fair models – ident: ref42 doi: 10.1109/QRS.2015.30 – start-page: 226 year: 2021 ident: ref31 article-title: Distribution-aware testing of neural networks using generative models publication-title: Proc IEEE/ACM 43rd Int Conf Softw Eng – year: 2020 ident: ref75 – ident: ref27 doi: 10.1609/aaai.v34i05.6267 – start-page: 1041 year: 2020 ident: ref11 article-title: Cats Are Not Fish: Deep Learning Testing Calls for Out-Of-Distribution Awareness publication-title: IEEE/ACM Int Conference on Automated Software Engineering (ASE) doi: 10.1145/3324884.3416609 – ident: ref74 doi: 10.1145/3194770.3194776 – ident: ref84 doi: 10.18653/v1/N18-2003 – ident: ref45 doi: 10.18653/v1/P16-2096 – ident: ref33 doi: 10.18653/v1/2021.acl-long.149 – year: 2020 ident: ref58 – ident: ref55 doi: 10.1017/S0269888921000102 – ident: ref9 doi: 10.1109/TSE.2014.2372785 – ident: ref8 doi: 10.1109/TSE.2021.3136169 – start-page: 962 year: 2017 ident: ref78 article-title: Fairness constraints: Mechanisms for fair classification publication-title: Proc 20th Int Conf Artif Intell Statist – volume: 1 year: 2019 ident: ref59 article-title: Language models are unsupervised multitask learners – ident: ref56 doi: 10.1145/945645.945658 – ident: ref61 doi: 10.1145/2939672.2939778 – year: 2020 ident: ref15 article-title: Sociolinguistically driven approaches for just natural language processing – ident: ref82 doi: 10.1145/3377811.3380331 – ident: ref68 doi: 10.1162/089120101753342653 – ident: ref14 doi: 10.1145/3468264.3468536 – ident: ref22 doi: 10.1145/3468264.3468537 – year: 2006 ident: ref24 article-title: Constructing grounded theory: A practical guide through qualitative analysis – start-page: 325 year: 2013 ident: ref79 article-title: Learning fair representations publication-title: Proc Int Conf Mach Learn – ident: ref85 doi: 10.18653/v1/D18-1521 – ident: ref70 doi: 10.1145/3377811.3380420 – ident: ref50 doi: 10.18653/v1/S18-2005 – year: 2019 ident: ref29 article-title: BERT: Pre-training of deep bidirectional transformers for language understanding – start-page: 4768 year: 2017 ident: ref52 article-title: A unified approach to interpreting model predictions publication-title: Proc 31st Int Conf Adv Neural Inf Process Syst 30 – year: 2020 ident: ref51 – ident: ref54 doi: 10.1609/aaai.v35i15.17594 – ident: ref4 doi: 10.1145/3133904 – ident: ref5 doi: 10.18653/v1/W19-3811 – ident: ref60 doi: 10.1007/s10603-005-3307-7 – ident: ref34 doi: 10.1145/230538.230561 – ident: ref30 doi: 10.1145/3173574.3173986 – ident: ref39 doi: 10.1080/01621459.1974.10482962 – start-page: 45 year: 2018 ident: ref43 article-title: Grammarinator: A grammar-based open source fuzzer publication-title: Proc 9th ACM SIGSOFT Int Workshop Automating TEST Case Des Selection Eval – ident: ref53 doi: 10.24963/ijcai.2020/64 – ident: ref16 doi: 10.18653/v1/2020.acl-main.485 – ident: ref83 doi: 10.18653/v1/D17-1323 – ident: ref86 doi: 10.18653/v1/D19-1531 – volume: 356 start-page: 183 year: 2017 ident: ref21 article-title: Semantics derived automatically from language corpora contain human-like biases publication-title: Science doi: 10.1126/science.aal4230 – year: 2017 ident: ref25 article-title: The trouble with bias publication-title: Proc Conf Neural Inf Process Syst – ident: ref80 doi: 10.1145/3278721.3278779 – ident: ref72 doi: 10.1145/3238147.3238165 – year: 2020 ident: ref71 article-title: Testing DNN image classifier for confusion & bias errors publication-title: Proc 42nd Int Conf Softw Eng – ident: ref41 doi: 10.1109/ASE.2019.00027 – year: 2021 ident: ref1 – year: 2017 ident: ref18 article-title: Racial disparity in natural language processing: A case study of social media african-american english – ident: ref64 doi: 10.2307/2291267 – ident: ref17 doi: 10.18653/v1/D16-1120 – ident: ref20 doi: 10.1109/ICDMW.2009.83 – ident: ref62 doi: 10.1609/aaai.v32i1.11491 – ident: ref3 doi: 10.1145/3338906.3338937 – ident: ref7 doi: 10.1145/1985793.1985795 – year: 2016 ident: ref6 publication-title: Machine Bias – ident: ref77 doi: 10.1145/3468264.3473117 – year: 2019 ident: ref26 article-title: Detecting bias with generative counterfactual face attribute augmentation – year: 2020 ident: ref66 – start-page: 992 year: 2011 ident: ref87 article-title: Handling conditional discrimination publication-title: Proc IEEE Int Conf Data Mining – ident: ref23 doi: 10.1145/3368089.3409697 – ident: ref81 doi: 10.1109/TSE.2019.2962027 – ident: ref35 doi: 10.1145/3106237.3106277 – ident: ref65 doi: 10.18653/v1/N18-2002
SSID	ssj0005775 ssib053395008
Score	2.5405598
Snippet	Software often produces biased outputs. In particular, machine learning (ML) based software is known to produce erroneous predictions when processing...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	5188
SubjectTerms	Bias Fault diagnosis Grammar Grammars Machine learning Natural language processing program debugging Sentiment analysis Software software fairness Software systems Software testing Task analysis Test pattern generators Testing Violations
Title	Astraea: Grammar-Based Fairness Testing
URI	https://ieeexplore.ieee.org/document/9678017 https://www.proquest.com/docview/2748563736
Volume	48
WOSCitedRecordID	wos000896667600025&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1939-3520 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0005775 issn: 0098-5589 databaseCode: RIE dateStart: 19750101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS8MwFH_M4cGLU6c4ndKDIIJ1aZY2ed6mbHqQIThlt5ImqQykk27z7zfpx0QUwVsPeaH8kryP5L33Azjrpwka6xb4GjX6VvtRH6lKfSExMJqkOij4U14e-HgsplN8bMDluhbGGFMkn5kr91m85eu5Wrmrsh5azWp30AZscB6VtVpf6Rych3V_zDAUWD9JEuxNnoY2EKTUxqfMWkvxzQQVnCo_FHFhXUat__3XDmxXXqQ3KJd9Fxom24NWzdDgVQe2DecDd5Nh5LV3l0tXpebfWKulvZGc5U7HeRPXZCN73Yfn0XBye-9X1Ai-ohgsHbI2nqTSIKFII6FZqgTTjjMskoEgEfKUijTSklNFApEmTJokpUmIoRLWqTuAZjbPzCF4BBULSZJoygzTilmHjhhtrGVHKxTQDvRqtGJV9Q139BVvcRE_EIwtvrHDN67w7cDFWuK97Jnxx9i2w3M9roKyA916QeLqUC1iG0CLMOrzfnT0u9QxbLm5y2yTLjSX-cqcwKb6WM4W-WmxXz4BkWm6sA
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3dS8MwED_mFPTFqVOcTu2DIIJ1aZa2Od-mbE6cQ3DK3kqbpDKQTrrNv9-kHxNRBN_6kCPll-Q-krv7AZy24wiVdgtsiRJtrf2ojVTENg_RUZLE0sn4U14G_nDIx2N8rMDFshZGKZUln6lL85m95cupWJirshZqzap30AqsuoxRkldrfSV0-L5bdsh0XY7loyTB1uipq0NBSnWEyrS95N-MUMaq8kMVZ_alV_vfn23BZuFHWp184behopIdqJUcDVZxZOtw1jF3GSq8sm7T0NSp2dfabkmrF05So-WskWmzkbzuwnOvO7rp2wU5gi0oOnODrY4oaaiQUKQelywWnEnDGuaFDice-jHlsSdDnwri8DhioYpiGrnoCq7duj2oJtNE7YNFUDCXRJGkTDEpmHbpiJJK23bUQg5tQKtEKxBF53BDYPEWZBEEwUDjGxh8gwLfBpwvJd7zrhl_jK0bPJfjCigb0CwXJCiO1SzQITR3vbbf9g5-lzqB9f7oYRAM7ob3h7Bh5slzT5pQnacLdQRr4mM-maXH2d75BHhBvfc
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Astraea%3A+Grammar-Based+Fairness+Testing&rft.jtitle=IEEE+transactions+on+software+engineering&rft.au=Soremekun%2C+Ezekiel&rft.au=Udeshi%2C+Sakshi&rft.au=Chattopadhyay%2C+Sudipta&rft.date=2022-12-01&rft.pub=IEEE&rft.issn=0098-5589&rft.volume=48&rft.issue=12&rft.spage=5188&rft.epage=5211&rft_id=info:doi/10.1109%2FTSE.2022.3141758&rft.externalDocID=9678017
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0098-5589&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0098-5589&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0098-5589&client=summon