Agreeing to disagree: active learning with noisy labels without crowdsourcing

We propose a new active learning method for classification, which handles label noise without relying on multiple oracles (i.e., crowdsourcing). We propose a strategy that selects (for labeling) instances with a high influence on the learned model. An instance x is said to have a high influence on t...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	International journal of machine learning and cybernetics Ročník 9; číslo 8; s. 1307 - 1319
Hlavní autori:	Bouguelia, Mohamed-Rafik, Nowaczyk, Slawomir, Santosh, K. C., Verikas, Antanas
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	Berlin/Heidelberg Springer Berlin Heidelberg 01.08.2018 Springer Nature B.V
Predmet:	Active learning Algorithms Artificial Intelligence Classification Complex Systems Computational Intelligence Control Crowdsourcing Data mining Datasets Engineering Interactive learning Label noise Labeling Labels Learning Machine learning Mechatronics Mislabeling Original Article Pattern Recognition Robotics Systems Biology Teaching methods Training Mislabeling Active learning Label noise Classification
ISSN:	1868-8071, 1868-808X, 1868-808X
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	We propose a new active learning method for classification, which handles label noise without relying on multiple oracles (i.e., crowdsourcing). We propose a strategy that selects (for labeling) instances with a high influence on the learned model. An instance x is said to have a high influence on the model h , if training h on x (with label y = h ( x ) ) would result in a model that greatly disagrees with h on labeling other instances. Then, we propose another strategy that selects (for labeling) instances that are highly influenced by changes in the learned model. An instance x is said to be highly influenced, if training h with a set of instances would result in a committee of models that agree on a common label for x but disagree with h ( x ). We compare the two strategies and we show, on different publicly available datasets, that selecting instances according to the first strategy while eliminating noisy labels according to the second strategy, greatly improves the accuracy compared to several benchmarking methods, even when a significant amount of instances are mislabeled.
AbstractList	We propose a new active learning method for classification, which handles label noise without relying on multiple oracles (i.e., crowdsourcing). We propose a strategy that selects (for labeling) instances with a high influence on the learned model. An instance x is said to have a high influence on the model h , if training h on x (with label y = h ( x )) would result in a model that greatly disagrees with h on labeling other instances. Then, we propose another strategy that selects (for labeling) instances that are highly influenced by changes in the learned model. An instance x is said to be highly influenced, if training h with a set of instances would result in a committee of models that agree on a common label for x but disagree with h ( x ). We compare the two strategies and we show, on different publicly available datasets, that selecting instances according to the first strategy while eliminating noisy labels according to the second strategy, greatly improves the accuracy compared to several benchmarking methods, even when a significant amount of instances are mislabeled. © Springer-Verlag Berlin Heidelberg 2017 We propose a new active learning method for classification, which handles label noise without relying on multiple oracles (i.e., crowdsourcing). We propose a strategy that selects (for labeling) instances with a high influence on the learned model. An instance x is said to have a high influence on the model h, if training h on x (with label y=h(x)) would result in a model that greatly disagrees with h on labeling other instances. Then, we propose another strategy that selects (for labeling) instances that are highly influenced by changes in the learned model. An instance x is said to be highly influenced, if training h with a set of instances would result in a committee of models that agree on a common label for x but disagree with h(x). We compare the two strategies and we show, on different publicly available datasets, that selecting instances according to the first strategy while eliminating noisy labels according to the second strategy, greatly improves the accuracy compared to several benchmarking methods, even when a significant amount of instances are mislabeled. We propose a new active learning method for classification, which handles label noise without relying on multiple oracles (i.e., crowdsourcing). We propose a strategy that selects (for labeling) instances with a high influence on the learned model. An instance x is said to have a high influence on the model h , if training h on x (with label y = h ( x ) ) would result in a model that greatly disagrees with h on labeling other instances. Then, we propose another strategy that selects (for labeling) instances that are highly influenced by changes in the learned model. An instance x is said to be highly influenced, if training h with a set of instances would result in a committee of models that agree on a common label for x but disagree with h ( x ). We compare the two strategies and we show, on different publicly available datasets, that selecting instances according to the first strategy while eliminating noisy labels according to the second strategy, greatly improves the accuracy compared to several benchmarking methods, even when a significant amount of instances are mislabeled.
Author	Verikas, Antanas Nowaczyk, Slawomir Bouguelia, Mohamed-Rafik Santosh, K. C.
Author_xml	– sequence: 1 givenname: Mohamed-Rafik surname: Bouguelia fullname: Bouguelia, Mohamed-Rafik organization: Center for Applied Intelligent Systems Research, Halmstad University – sequence: 2 givenname: Slawomir surname: Nowaczyk fullname: Nowaczyk, Slawomir organization: Center for Applied Intelligent Systems Research, Halmstad University – sequence: 3 givenname: K. C. orcidid: 0000-0003-4176-0236 surname: Santosh fullname: Santosh, K. C. email: santosh.kc@usd.edu organization: Department of Computer Science, The University of South Dakota – sequence: 4 givenname: Antanas surname: Verikas fullname: Verikas, Antanas organization: Center for Applied Intelligent Systems Research, Halmstad University
BackLink	https://urn.kb.se/resolve?urn=urn:nbn:se:hh:diva-33365$$DView record from Swedish Publication Index (Högskolan i Halmstad)
BookMark	eNp9UMtOwzAQtFCRKKUfwC0SRxTwI3FsblV5SkVcAHGzHNdpXYW42A5V_x6HVEVCAh_sXe_M7uwcg0FjGw3AKYIXCMLi0iMCM5xCVKSQZnkKD8AQMcpSBtnbYB8X6AiMvV_BeCgkBOIheJwsnNamWSTBJnPjZZdeJVIF86mTWkvXdMWNCcukscZvk1qWuvbfP7YNiXJ2M_e2dSriTsBhJWuvx7t3BF5ub56n9-ns6e5hOpmlivA8pCznWUYzjWhZIVZKpDjEuERzklNaqVjCUGOUl1JnTCnOOKo4xxjmCpXxIiNw3vf1G71uS7F25l26rbDSiGvzOhHWLcRyKQghNI_osx69dvaj1T6IVdTbRIECc8QJzSgtIgr1qLiR905X-64Iis5m0dssos2is1nAyCl-cZQJMhjbBCdN_S8T7zaIU5qFdj-a_iZ9AU1Ekoo
CitedBy_id	crossref_primary_10_1145_3473910 crossref_primary_10_1109_ACCESS_2020_3034032 crossref_primary_10_1007_s13042_019_00928_3 crossref_primary_10_1007_s11277_023_10802_9 crossref_primary_10_1007_s13042_024_02135_1 crossref_primary_10_1007_s00371_021_02094_6 crossref_primary_10_1109_TPAMI_2021_3120012 crossref_primary_10_3390_ijerph18062842 crossref_primary_10_1109_JBHI_2020_3008601 crossref_primary_10_1038_s41598_025_10942_x crossref_primary_10_1007_s10772_018_09559_5 crossref_primary_10_1007_s10994_020_05897_1 crossref_primary_10_1007_s11277_023_10371_x crossref_primary_10_1007_s13042_019_00991_w crossref_primary_10_3389_fcomp_2019_00004 crossref_primary_10_1007_s10772_018_09560_y crossref_primary_10_1007_s10772_018_09562_w crossref_primary_10_1145_3519311 crossref_primary_10_1007_s10772_018_09582_6 crossref_primary_10_1007_s10772_018_9542_5 crossref_primary_10_1007_s10772_018_9544_3 crossref_primary_10_1007_s10772_018_9545_2 crossref_primary_10_1007_s10772_018_9525_6 crossref_primary_10_1007_s11042_019_08286_6 crossref_primary_10_3390_chemosensors9040078 crossref_primary_10_1007_s10916_020_01562_1 crossref_primary_10_1145_3551385 crossref_primary_10_1007_s11277_021_08736_1 crossref_primary_10_1007_s13042_018_0883_9 crossref_primary_10_1016_j_comnet_2021_108227 crossref_primary_10_1007_s00521_019_04468_3 crossref_primary_10_1109_JTEHM_2020_3012926 crossref_primary_10_1016_j_psep_2020_11_007 crossref_primary_10_3390_jimaging5010002 crossref_primary_10_1007_s13042_019_00932_7 crossref_primary_10_1109_ACCESS_2021_3088901 crossref_primary_10_1007_s10772_018_9550_5 crossref_primary_10_1007_s10772_018_9552_3 crossref_primary_10_1016_j_procs_2020_03_268 crossref_primary_10_1007_s10772_018_09578_2 crossref_primary_10_1016_j_procs_2020_03_327 crossref_primary_10_1007_s10772_019_09603_y crossref_primary_10_1007_s10772_018_9534_5 crossref_primary_10_1007_s10772_018_9536_3 crossref_primary_10_1016_j_patcog_2018_10_006 crossref_primary_10_1007_s10462_021_10021_3 crossref_primary_10_1007_s10772_018_9556_z
Cites_doi	10.1109/TNNLS.2013.2292894 10.1007/s10618-013-0306-1 10.1016/j.patrec.2015.11.010 10.1147/JRD.2015.2390017 10.1109/TCYB.2014.2344674 10.1007/s13042-010-0003-y 10.1109/TNNLS.2015.2401595 10.2200/S00429ED1V01Y201207AIM018 10.1016/j.patrec.2013.10.011 10.1007/s13042-014-0239-z 10.1007/s10618-016-0469-7 10.1002/widm.1132 10.1109/TGRS.2012.2203605 10.1109/ICDM.2012.162 10.1007/s13042-015-0458-y 10.1109/ICDAR.2013.171 10.1109/IJCNN.2014.6889572 10.1007/978-3-319-27677-9_3 10.1109/ICDM.2013.15 10.21437/Interspeech.2012-131 10.1007/978-3-642-33715-4_36 10.1007/s13042-014-0275-8 10.1109/ICDAR.2015.7333767 10.1007/978-3-540-78646-7_34
ContentType	Journal Article
Copyright	Springer-Verlag Berlin Heidelberg 2017 Springer-Verlag Berlin Heidelberg 2017.
Copyright_xml	– notice: Springer-Verlag Berlin Heidelberg 2017 – notice: Springer-Verlag Berlin Heidelberg 2017.
DBID	AAYXX CITATION 8FE 8FG ABJCF AFKRA ARAPS AZQEC BENPR BGLVJ CCPQU DWQXO GNUQQ HCIFZ JQ2 K7- L6V M7S P5Z P62 PHGZM PHGZT PKEHL PQEST PQGLB PQQKQ PQUKI PRINS PTHSS AAXBQ ADTPV AOWAS D8T D8Z ZZAVC
DOI	10.1007/s13042-017-0645-0
DatabaseName	CrossRef ProQuest SciTech Collection ProQuest Technology Collection ProQuest Materials Science & Engineering ProQuest Central UK/Ireland Advanced Technologies & Computer Science Collection ProQuest Central Essentials ProQuest Central Technology Collection ProQuest One Community College ProQuest Central ProQuest Central Student SciTech Premium Collection ProQuest Computer Science Collection Computer Science Database (ProQuest) ProQuest Engineering Collection Engineering Database (subscription) Advanced Technologies & Aerospace Database ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Premium ProQuest One Academic ProQuest One Academic Middle East (New) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Applied & Life Sciences ProQuest One Academic (retired) ProQuest One Academic UKI Edition ProQuest Central China Engineering collection SWEPUB Högskolan i Halmstad full text SwePub SwePub Articles SWEPUB Freely available online SWEPUB Högskolan i Halmstad SwePub Articles full text
DatabaseTitle	CrossRef Computer Science Database ProQuest Central Student Technology Collection ProQuest One Academic Middle East (New) ProQuest Advanced Technologies & Aerospace Collection ProQuest Central Essentials ProQuest Computer Science Collection SciTech Premium Collection ProQuest One Community College ProQuest Central China ProQuest Central ProQuest One Applied & Life Sciences ProQuest Engineering Collection ProQuest Central Korea ProQuest Central (New) Engineering Collection Advanced Technologies & Aerospace Collection Engineering Database ProQuest One Academic Eastern Edition ProQuest Technology Collection ProQuest SciTech Collection Advanced Technologies & Aerospace Database ProQuest One Academic UKI Edition Materials Science & Engineering Collection ProQuest One Academic ProQuest One Academic (New)
DatabaseTitleList	Computer Science Database
Database_xml	– sequence: 1 dbid: BENPR name: ProQuest Central url: https://www.proquest.com/central sourceTypes: Aggregation Database
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering Sciences (General)
EISSN	1868-808X
EndPage	1319
ExternalDocumentID	oai_DiVA_org_hh_33365 10_1007_s13042_017_0645_0
GroupedDBID	-EM 06D 0R~ 0VY 1N0 203 29~ 2JY 2VQ 30V 4.4 406 408 409 40D 96X AACDK AAHNG AAIAL AAJBT AAJKR AANZL AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYQN AAYTO AAYZH AAZMS ABAKF ABBXA ABDZT ABECU ABFTD ABFTV ABHQN ABJCF ABJNI ABJOX ABKCH ABMQK ABQBU ABSXP ABTEG ABTHY ABTKH ABTMW ABULA ABWNU ABXPI ACAOD ACDTI ACGFS ACHSB ACKNC ACMLO ACOKC ACPIV ACZOJ ADHHG ADHIR ADINQ ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF ADZKW AEBTG AEFQL AEGNC AEJHL AEJRE AEMSY AENEX AEOHA AEPYU AESKC AETCA AEVLU AEXYK AFBBN AFKRA AFLOW AFQWF AFWTZ AFZKB AGAYW AGDGC AGJBK AGMZJ AGQEE AGQMX AGRTI AGWZB AGYKE AHAVH AHBYD AHKAY AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ AKLTO ALFXC ALMA_UNASSIGNED_HOLDINGS AMKLP AMXSW AMYLF AMYQR ANMIH ARAPS AUKKA AXYYD AYJHY BENPR BGLVJ BGNMA CCPQU CSCUP DNIVK DPUIP EBLON EBS EIOEI EJD ESBYG FERAY FIGPU FINBP FNLPD FRRFC FSGXE FYJPI GGCAI GGRSB GJIRD GQ6 GQ7 GQ8 H13 HCIFZ HMJXF HQYDN HRMNR HZ~ I0C IKXTQ IWAJR IXD IZIGR J-C J0Z JBSCW JCJTX JZLTJ K7- KOV LLZTM M4Y M7S NPVJJ NQJWS NU0 O9- O93 O9J P2P P9P PT4 PTHSS QOS R89 R9I RLLFE ROL RSV S27 S3B SEG SHX SISQX SJYHP SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE T13 TSG U2A UG4 UOJIU UTJUX UZXMN VC2 VFIZW W48 WK8 Z45 Z7X Z83 Z88 ZMTXR ~A9 AAYXX ABBRH ABDBE ABFSG ABRTQ ACSTC ADKFA AEZWR AFDZB AFFHD AFHIU AFOHR AHPBZ AHWEU AIXLP ATHPR AYFIA CITATION PHGZM PHGZT PQGLB 8FE 8FG AZQEC DWQXO GNUQQ JQ2 L6V P62 PKEHL PQEST PQQKQ PQUKI PRINS AAXBQ ADTPV AOWAS D8T D8Z ZZAVC
ID	FETCH-LOGICAL-c395t-8594464e16bf18ba1c9022b1d3566fc46420e215bae48cc9891f992205c1b05c3
IEDL.DBID	RSV
ISICitedReferencesCount	61
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000438855100006&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1868-8071 1868-808X
IngestDate	Tue Nov 04 15:45:39 EST 2025 Wed Nov 05 06:26:05 EST 2025 Tue Nov 18 21:42:23 EST 2025 Sat Nov 29 05:59:38 EST 2025 Fri Feb 21 02:41:03 EST 2025
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	8
Keywords	Mislabeling Active learning Label noise Classification
Language	English
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c395t-8594464e16bf18ba1c9022b1d3566fc46420e215bae48cc9891f992205c1b05c3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0003-4176-0236
OpenAccessLink	https://urn.kb.se/resolve?urn=urn:nbn:se:hh:diva-33365
PQID	2919364667
PQPubID	2043904
PageCount	13
ParticipantIDs	swepub_primary_oai_DiVA_org_hh_33365 proquest_journals_2919364667 crossref_primary_10_1007_s13042_017_0645_0 crossref_citationtrail_10_1007_s13042_017_0645_0 springer_journals_10_1007_s13042_017_0645_0
PublicationCentury	2000
PublicationDate	2018-08-01
PublicationDateYYYYMMDD	2018-08-01
PublicationDate_xml	– month: 08 year: 2018 text: 2018-08-01 day: 01
PublicationDecade	2010
PublicationPlace	Berlin/Heidelberg
PublicationPlace_xml	– name: Berlin/Heidelberg – name: Heidelberg
PublicationTitle	International journal of machine learning and cybernetics
PublicationTitleAbbrev	Int. J. Mach. Learn. & Cyber
PublicationYear	2018
Publisher	Springer Berlin Heidelberg Springer Nature B.V
Publisher_xml	– name: Springer Berlin Heidelberg – name: Springer Nature B.V
References	Gilad-Bachrach, Navot, Tishby (CR10) 2005; 5 Ipeirotis, Provost, Sheng, Wang (CR13) 2014; 28 Bouguelia, Belaid, Belaid (CR5) 2016; 70 Zhang, Wang, Yun (CR33) 2015; 26 Abedini, Codella, Connell, Garnavi, Merler, Pankanti, Smith, Syeda-Mahmood (CR1) 2015; 59 CR17 CR16 Wu, Lin, Weng (CR31) 2004; 5 CR15 CR12 Tuia, Munoz-Mari (CR28) 2013; 51 CR30 Settles, Craven, Ray (CR25) 2008; 20 Hamidzadeh, Monsefi, Yazdi (CR11) 2016; 7 Ramirez-Loaiza, Sharma, Kumar, Bilgic (CR19) 2016; 31 Ren, Li (CR21) 2015 CR2 Zhang, Wu, Shengs (CR32) 2015; 45 CR4 CR3 Pedregosa, Varoquaux, Gramfort, Michel, Thirion, Grisel, Blondel, Prettenhofer, Weiss, Dubourg, Vanderplas, Passos, Cournapeau, Brucher, Perrot, Duchesnay (CR18) 2011; 12 CR7 CR29 Settles (CR24) 2012; 6 CR26 CR23 CR22 CR20 Small, Roth (CR27) 2010; 1 Fang, Zhu (CR8) 2014; 43 Kremer, Steenstrup Pedersen, Igel (CR14) 2014; 4 Frnay, Verleysen (CR9) 2014; 25 Bouneffouf, Laroche, Urvoy, Fraud, Allesiardo (CR6) 2014; 26 645_CR7 D Tuia (645_CR28) 2013; 51 MR Bouguelia (645_CR5) 2016; 70 J Kremer (645_CR14) 2014; 4 ME Ramirez-Loaiza (645_CR19) 2016; 31 B Frnay (645_CR9) 2014; 25 W Ren (645_CR21) 2015 645_CR16 645_CR17 645_CR15 645_CR12 J Hamidzadeh (645_CR11) 2016; 7 645_CR30 M Abedini (645_CR1) 2015; 59 PG Ipeirotis (645_CR13) 2014; 28 XY Zhang (645_CR33) 2015; 26 TF Wu (645_CR31) 2004; 5 B Settles (645_CR25) 2008; 20 645_CR29 D Bouneffouf (645_CR6) 2014; 26 B Settles (645_CR24) 2012; 6 645_CR26 M Fang (645_CR8) 2014; 43 645_CR23 645_CR2 645_CR22 645_CR20 F Pedregosa (645_CR18) 2011; 12 J Zhang (645_CR32) 2015; 45 R Gilad-Bachrach (645_CR10) 2005; 5 K Small (645_CR27) 2010; 1 645_CR4 645_CR3
References_xml	– ident: CR22 – volume: 25 start-page: 845 issue: 5 year: 2014 end-page: 869 ident: CR9 article-title: Classification in the presence of label noise: a survey publication-title: IEEE Trans Neural Netw Learn Syst doi: 10.1109/TNNLS.2013.2292894 – volume: 28 start-page: 402 issue: 2 year: 2014 end-page: 441 ident: CR13 article-title: Repeated labeling using multiple noisy labelers publication-title: Data Min Knowl Discov doi: 10.1007/s10618-013-0306-1 – ident: CR4 – ident: CR2 – ident: CR16 – volume: 12 start-page: 2825 year: 2011 end-page: 2830 ident: CR18 article-title: Scikit-learn: machine learning in python publication-title: J Mach Learn Res – ident: CR12 – ident: CR30 – volume: 70 start-page: 38 year: 2016 end-page: 44 ident: CR5 article-title: An adaptive streaming active learning strategy based on instance weighting publication-title: Pattern Recogn Lett doi: 10.1016/j.patrec.2015.11.010 – ident: CR29 – volume: 59 start-page: 1 issue: 2/3 year: 2015 end-page: 18 ident: CR1 article-title: A generalized framework for medical image classification and recognition publication-title: IBM J Res Dev doi: 10.1147/JRD.2015.2390017 – volume: 45 start-page: 1095 issue: 5 year: 2015 end-page: 1107 ident: CR32 article-title: Active learning with imbalanced multiple noisy labeling publication-title: IEEE Trans Cybern doi: 10.1109/TCYB.2014.2344674 – volume: 1 start-page: 3 issue: 1–4 year: 2010 end-page: 25 ident: CR27 article-title: Margin-based active learning for structured predictions publication-title: Int J Mach Learn Cybern doi: 10.1007/s13042-010-0003-y – volume: 26 start-page: 3034 issue: 12 year: 2015 end-page: 3044 ident: CR33 article-title: Bidirectional active learning: a two-way exploration into unlabeled and labeled data set publication-title: IEEE Trans Neural Netw Learn Syst doi: 10.1109/TNNLS.2015.2401595 – volume: 5 start-page: 443 year: 2005 end-page: 450 ident: CR10 article-title: Query by committee made real publication-title: Adv Neural Inf Process Syst – ident: CR23 – volume: 6 start-page: 1 issue: 1 year: 2012 end-page: 114 ident: CR24 article-title: Active learning publication-title: Synth Lect Artif Intell Mach Learn doi: 10.2200/S00429ED1V01Y201207AIM018 – volume: 43 start-page: 98 year: 2014 end-page: 108 ident: CR8 article-title: Active learning with uncertain labeling knowledge publication-title: Pattern Recogn Lett doi: 10.1016/j.patrec.2013.10.011 – volume: 7 start-page: 25 issue: 1 year: 2016 end-page: 45 ident: CR11 article-title: Large symmetric margin instance selection algorithm publication-title: Int J Mach Learn Cybern doi: 10.1007/s13042-014-0239-z – volume: 20 start-page: 1289 year: 2008 end-page: 1296 ident: CR25 article-title: Multiple-instance active learning publication-title: Adv Neural Inf Process Syst – ident: CR3 – ident: CR15 – volume: 26 start-page: 405 issue: 12 year: 2014 end-page: 412 ident: CR6 article-title: Contextual bandit for active learning: active thompson sampling publication-title: Int Conf Neural Inf Process – ident: CR17 – year: 2015 ident: CR21 article-title: Graph based semi-supervised learning via label fitting publication-title: Int J Mach Learn Cybern – volume: 31 start-page: 287 year: 2016 end-page: 313 ident: CR19 article-title: Active learning: an empirical study of common baselines publication-title: Data Min Knowl Discove doi: 10.1007/s10618-016-0469-7 – ident: CR7 – volume: 5 start-page: 975 year: 2004 end-page: 1005 ident: CR31 article-title: Probability estimates for multi-class classification by pairwise coupling publication-title: J Mach Learn Res – volume: 4 start-page: 313 issue: 4 year: 2014 end-page: 326 ident: CR14 article-title: Active learning with support vector machines publication-title: Wiley Interdiscip Rev Data Min Knowl Discov doi: 10.1002/widm.1132 – ident: CR26 – volume: 51 start-page: 872 issue: 2 year: 2013 end-page: 880 ident: CR28 article-title: Learning user’s confidence for active learning publication-title: IEEE Trans Geosci Remote Sens doi: 10.1109/TGRS.2012.2203605 – ident: CR20 – ident: 645_CR20 doi: 10.1109/ICDM.2012.162 – ident: 645_CR23 – volume: 12 start-page: 2825 year: 2011 ident: 645_CR18 publication-title: J Mach Learn Res – volume: 51 start-page: 872 issue: 2 year: 2013 ident: 645_CR28 publication-title: IEEE Trans Geosci Remote Sens doi: 10.1109/TGRS.2012.2203605 – volume: 45 start-page: 1095 issue: 5 year: 2015 ident: 645_CR32 publication-title: IEEE Trans Cybern doi: 10.1109/TCYB.2014.2344674 – volume: 28 start-page: 402 issue: 2 year: 2014 ident: 645_CR13 publication-title: Data Min Knowl Discov doi: 10.1007/s10618-013-0306-1 – volume: 26 start-page: 3034 issue: 12 year: 2015 ident: 645_CR33 publication-title: IEEE Trans Neural Netw Learn Syst doi: 10.1109/TNNLS.2015.2401595 – ident: 645_CR3 – volume: 4 start-page: 313 issue: 4 year: 2014 ident: 645_CR14 publication-title: Wiley Interdiscip Rev Data Min Knowl Discov doi: 10.1002/widm.1132 – ident: 645_CR17 – year: 2015 ident: 645_CR21 publication-title: Int J Mach Learn Cybern doi: 10.1007/s13042-015-0458-y – ident: 645_CR2 doi: 10.1109/ICDAR.2013.171 – ident: 645_CR30 doi: 10.1109/IJCNN.2014.6889572 – volume: 59 start-page: 1 issue: 2/3 year: 2015 ident: 645_CR1 publication-title: IBM J Res Dev doi: 10.1147/JRD.2015.2390017 – volume: 70 start-page: 38 year: 2016 ident: 645_CR5 publication-title: Pattern Recogn Lett doi: 10.1016/j.patrec.2015.11.010 – volume: 25 start-page: 845 issue: 5 year: 2014 ident: 645_CR9 publication-title: IEEE Trans Neural Netw Learn Syst doi: 10.1109/TNNLS.2013.2292894 – ident: 645_CR4 doi: 10.1007/978-3-319-27677-9_3 – ident: 645_CR26 doi: 10.1109/ICDM.2013.15 – ident: 645_CR22 doi: 10.21437/Interspeech.2012-131 – volume: 1 start-page: 3 issue: 1–4 year: 2010 ident: 645_CR27 publication-title: Int J Mach Learn Cybern doi: 10.1007/s13042-010-0003-y – ident: 645_CR29 doi: 10.1007/978-3-642-33715-4_36 – ident: 645_CR7 doi: 10.1007/s13042-014-0275-8 – volume: 5 start-page: 975 year: 2004 ident: 645_CR31 publication-title: J Mach Learn Res – volume: 20 start-page: 1289 year: 2008 ident: 645_CR25 publication-title: Adv Neural Inf Process Syst – volume: 5 start-page: 443 year: 2005 ident: 645_CR10 publication-title: Adv Neural Inf Process Syst – volume: 43 start-page: 98 year: 2014 ident: 645_CR8 publication-title: Pattern Recogn Lett doi: 10.1016/j.patrec.2013.10.011 – volume: 7 start-page: 25 issue: 1 year: 2016 ident: 645_CR11 publication-title: Int J Mach Learn Cybern doi: 10.1007/s13042-014-0239-z – ident: 645_CR12 doi: 10.1109/ICDAR.2015.7333767 – volume: 31 start-page: 287 year: 2016 ident: 645_CR19 publication-title: Data Min Knowl Discove doi: 10.1007/s10618-016-0469-7 – ident: 645_CR16 – volume: 26 start-page: 405 issue: 12 year: 2014 ident: 645_CR6 publication-title: Int Conf Neural Inf Process – volume: 6 start-page: 1 issue: 1 year: 2012 ident: 645_CR24 publication-title: Synth Lect Artif Intell Mach Learn doi: 10.2200/S00429ED1V01Y201207AIM018 – ident: 645_CR15 doi: 10.1007/978-3-540-78646-7_34
SSID	ssj0000603302 ssib031263576 ssib033405570
Score	2.44115
Snippet	We propose a new active learning method for classification, which handles label noise without relying on multiple oracles (i.e., crowdsourcing). We propose a...
SourceID	swepub proquest crossref springer
SourceType	Open Access Repository Aggregation Database Enrichment Source Index Database Publisher
StartPage	1307
SubjectTerms	Active learning Algorithms Artificial Intelligence Classification Complex Systems Computational Intelligence Control Crowdsourcing Data mining Datasets Engineering Interactive learning Label noise Labeling Labels Learning Machine learning Mechatronics Mislabeling Original Article Pattern Recognition Robotics Systems Biology Teaching methods Training
SummonAdditionalLinks	– databaseName: ProQuest Central dbid: BENPR link: http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1Lb9QwELZK2wMXoDzUhYJ86AGoLOzY8dpc0EJb9dJVhaDqzYodp10JbcomBfHvmUmcjcphL1wiJXbiODOep_MNIYech0rbSjNYDYopWWbMWp8z43mpQznNouddsYnpfG6uruxFCrg1aVvlIBM7QV3WAWPkHzILpoZWWk8_3f5kWDUKs6uphMYDsoNIZcDnO59P5hdfB46SArFWRoUrpeowp9ZRGK7hWr8x0WiDyLxiSH12_9ehs89QkCOsG-P3lddoka6TqP8AjnZK6vTx_07vCXmUzFM66_lpj2zF5VOylwRAQ98mlOp3z8j5DDz1CK9D25qWi6bA04-06OQnTcUorinGeemyXjR_KPAbKOLuSn3XUpji77LLHUC_5-T76cm3L2cs1WZgQdq8ZSa34EiqKLSvhPGFCBasAS9KCfZhFaAp4xHMCV9EZUKwxooKIXB5HoSHg3xBtpf1Mu4TqkWV2VhFkCy5KsEdw9RdoRSP3meBhwnhw0d3IQGXY_2MH26EXEY6OaCTQzo5PiHv17fc9qgdmzofDCRxaQE3bqTHhBwN1B2bNzzssGeA9biI3328uJy5enXtbm6clFLnLzeP-Yo8BMPM9BsND8h2u7qLr8lu-NUumtWbxOR_Ac9C_RM priority: 102 providerName: ProQuest
Title	Agreeing to disagree: active learning with noisy labels without crowdsourcing
URI	https://link.springer.com/article/10.1007/s13042-017-0645-0 https://www.proquest.com/docview/2919364667 https://urn.kb.se/resolve?urn=urn:nbn:se:hh:diva-33365
Volume	9
WOSCitedRecordID	wos000438855100006&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVPQU databaseName: Advanced Technologies & Aerospace Database customDbUrl: eissn: 1868-808X dateEnd: 20241214 omitProxy: false ssIdentifier: ssj0000603302 issn: 1868-808X databaseCode: P5Z dateStart: 20101201 isFulltext: true titleUrlDefault: https://search.proquest.com/hightechjournals providerName: ProQuest – providerCode: PRVPQU databaseName: Computer Science Database (ProQuest) customDbUrl: eissn: 1868-808X dateEnd: 20241214 omitProxy: false ssIdentifier: ssj0000603302 issn: 1868-808X databaseCode: K7- dateStart: 20101201 isFulltext: true titleUrlDefault: http://search.proquest.com/compscijour providerName: ProQuest – providerCode: PRVPQU databaseName: Engineering Database (subscription) customDbUrl: eissn: 1868-808X dateEnd: 20241214 omitProxy: false ssIdentifier: ssj0000603302 issn: 1868-808X databaseCode: M7S dateStart: 20101201 isFulltext: true titleUrlDefault: http://search.proquest.com providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 1868-808X dateEnd: 20241214 omitProxy: false ssIdentifier: ssj0000603302 issn: 1868-808X databaseCode: BENPR dateStart: 20101201 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVAVX databaseName: SpringerLink Contemporary customDbUrl: eissn: 1868-808X dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0000603302 issn: 1868-808X databaseCode: RSV dateStart: 20101201 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3dbxQhEJ9o64M-VFs1ntYLD33wIySwsBz07dQ2JsbLxdam8YUsLNteYm7N7VbT_77DftzWpmmiLyQLLASYGWYY-A3AHmO-UKZQFLlBUinyhBrjUqody5XPJ0lwrAk2MZnN9OmpmXfvuKv-tnvvkmwk9fDYLVreNErViLFG0U7fTCPYTDTRj056IhI8wqsMe6wQsoGZWh-8MIV57V1ErXQE4-W9t_O2Xv7erwYldO03vYEx2uxLh4__a0RPYKtTQ8m0pZttuBeWO_DoGjjhDmx3bF-RNx029dun8HWK9nnAclKXJF9UWfzcJ1kjNUkXguKMxNNdsiwX1SVBKsPtt8kpL2qCo_yTNx4DrPcMvh8eHH_8TLuIDNQLk9ZUpwbNRxm4cgXXLuPeoA7geC5QKyw8FiUsoBLhsiC190YbXkTgW5Z67jARz2FjWS7DCyCKF4kJRUB5ksocjbDosMukZMG5xDM_AtbPu_UdXHmMmvHTDkDLcfosTp-N02fZCN6tf_nVYnXcVXm3X0zbsW1lE4P6rJJKTUbwvl-zofiOxvZaGlj3G1G7Py1OprZcndnzcyuEUOnLf2r0FTxE7Uy3tw13YaNeXYTX8MD_rhfVagybHw5m829juP9lQsfxyuoRpvP0x7hhhysxu_nX
linkProvider	Springer Nature
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1V1Lb9QwEB6VggQXSoGKhQI-FImHrDqx47UrIbSiVK22XXEoqDc3dpztSmhTNilV_xS_kXEeG8Fhbz1wiRTHiRX7m4c99jcAO4y5XOpcUpQGQQXPYqq1TaiyLJMuG8besjrZxHAyUWdn-usa_O7OwoRtlZ1OrBV1VriwRr4ba3Q1pJBy-OnyJw1Zo0J0tUuh0cBi7G-uccpWfjzax_F9E8cHX04_H9I2qwB1XCcVVYnGKZDwkbR5pGwaOY12zEYZR88md_goZh4NoU29UM5ppaM8kLeyxEUWLxy_ewfuCq5kkKjxkHb45VFgdunNO-eiZrharvkwiWXNNkglVeABjrpAa32aLywt0GA2AokcZX-byt7_XYZs_6E3rU3iwcb_1pmP4GHrfJNRIy2bsObnj2GzVW8ledtycL97Aiej6cJ7_H1SFSSblWm43SNpbR1Im2pjSsIqNpkXs_KGoDShm1GXFFcVwS69zurICNZ7Ct9u5be2YH1ezP0zIDLKY-1zj3ozERlONkNgMhWCeWtjx9wAWDfIxrW07CE7yA_TE0oHXBjEhQm4MGwA75evXDacJKsqb3cQMK16Kk0__gP40KGpf7ziYzsN4JbtBnby_dn3kSkWU3NxYTjnMnm-us3XcP_w9OTYHB9Nxi_gAbqgqtlSuQ3r1eLKv4R77lc1KxevavEicH7bEPwDsHVV7A
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1Lb9QwEB6hghAcgBYqlrbgQw88ZNWOHa_d24qyAgGrikfVmxU7TrsSSqpNCuLfM85jQxGqhLhEiu3Esj2eh2f8DcA-Y75QplAUd4OkUuQJNcalVDuWK59Pk-BYm2xiuljo01Nz3Oc5rYdo98El2d1piChNZXNwkRcH48W3aIXTyGEj3hpFm_2mREMmxnR9-nwyEJTgEWpllLdCyBZyan0IwxSWdXGJWukIzMsHz-fferkqu0aFdO1D_QNvtJVR8_v_PboHcK9XT8mso6dNuBHKLbj7G2jhFmz27KAmz3vM6hcP4eMM7faA9aSpSL6ss_h6SLKWm5I-NcUZiae-pKyW9U-C1IdiuS2pLhuCI_6Rt54EbPcIvs7ffHn9lvaZGqgXJm2oTg2alTJw5QquXca9Qd3A8Vygtlh4rEpYQOXCZUFq7402vIiAuCz13OFDbMNGWZXhMRDFi8SEIiCfSWWOxll05GVSsuBc4pmfABvWwPoexjxm0_hmRwDmOH0Wp8_G6bNsAi_Xn1x0GB7XNd4dFtb227m2iUE9V0mlphN4NazfWH3Nz_Y7elj3G9G8j5YnM1utzuz5uRVCqPTJP_30Gdw-PprbD-8W73fgDipwugtI3IWNZnUZ9uCW_94s69XTdi_8As3tAUw
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Agreeing+to+disagree%3A+active+learning+with+noisy+labels+without+crowdsourcing&rft.jtitle=International+journal+of+machine+learning+and+cybernetics&rft.au=Bouguelia%2C+Mohamed-Rafik&rft.au=Nowaczyk%2C+Slawomir&rft.au=Santosh%2C+K.+C.&rft.au=Verikas%2C+Antanas&rft.date=2018-08-01&rft.issn=1868-8071&rft.eissn=1868-808X&rft.volume=9&rft.issue=8&rft.spage=1307&rft.epage=1319&rft_id=info:doi/10.1007%2Fs13042-017-0645-0&rft.externalDBID=n%2Fa&rft.externalDocID=10_1007_s13042_017_0645_0
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1868-8071&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1868-8071&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1868-8071&client=summon