Active Learning of Classification Models with Likert-Scale Feedback

Annotation of classification data by humans can be a time-consuming and tedious process. Finding ways of reducing the annotation effort is critical for building the classification models in practice and for applying them to a variety of classification tasks. In this paper, we develop a new active le...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings of the ... SIAM International Conference on Data Mining Vol. 2017; p. 28
Main Authors: Xue, Yanbing, Hauskrecht, Milos
Format: Journal Article
Language:English
Published: United States 2017
ISSN:2167-0102
Online Access:Get more information
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Annotation of classification data by humans can be a time-consuming and tedious process. Finding ways of reducing the annotation effort is critical for building the classification models in practice and for applying them to a variety of classification tasks. In this paper, we develop a new active learning framework that combines two strategies to reduce the annotation effort. First, it relies on label uncertainty information obtained from the human in terms of the Likert-scale feedback. Second, it uses active learning to annotate examples with the greatest expected change. We propose a Bayesian approach to calculate the expectation and an incremental SVM solver to reduce the time complexity of the solvers. We show the combination of our active learning strategy and the Likert-scale feedback can learn classification models more rapidly and with a smaller number of labeled instances than methods that rely on either Likert-scale labels or active learning alone.
AbstractList Annotation of classification data by humans can be a time-consuming and tedious process. Finding ways of reducing the annotation effort is critical for building the classification models in practice and for applying them to a variety of classification tasks. In this paper, we develop a new active learning framework that combines two strategies to reduce the annotation effort. First, it relies on label uncertainty information obtained from the human in terms of the Likert-scale feedback. Second, it uses active learning to annotate examples with the greatest expected change. We propose a Bayesian approach to calculate the expectation and an incremental SVM solver to reduce the time complexity of the solvers. We show the combination of our active learning strategy and the Likert-scale feedback can learn classification models more rapidly and with a smaller number of labeled instances than methods that rely on either Likert-scale labels or active learning alone.Annotation of classification data by humans can be a time-consuming and tedious process. Finding ways of reducing the annotation effort is critical for building the classification models in practice and for applying them to a variety of classification tasks. In this paper, we develop a new active learning framework that combines two strategies to reduce the annotation effort. First, it relies on label uncertainty information obtained from the human in terms of the Likert-scale feedback. Second, it uses active learning to annotate examples with the greatest expected change. We propose a Bayesian approach to calculate the expectation and an incremental SVM solver to reduce the time complexity of the solvers. We show the combination of our active learning strategy and the Likert-scale feedback can learn classification models more rapidly and with a smaller number of labeled instances than methods that rely on either Likert-scale labels or active learning alone.
Annotation of classification data by humans can be a time-consuming and tedious process. Finding ways of reducing the annotation effort is critical for building the classification models in practice and for applying them to a variety of classification tasks. In this paper, we develop a new active learning framework that combines two strategies to reduce the annotation effort. First, it relies on label uncertainty information obtained from the human in terms of the Likert-scale feedback. Second, it uses active learning to annotate examples with the greatest expected change. We propose a Bayesian approach to calculate the expectation and an incremental SVM solver to reduce the time complexity of the solvers. We show the combination of our active learning strategy and the Likert-scale feedback can learn classification models more rapidly and with a smaller number of labeled instances than methods that rely on either Likert-scale labels or active learning alone.
Author Hauskrecht, Milos
Xue, Yanbing
Author_xml – sequence: 1
  givenname: Yanbing
  surname: Xue
  fullname: Xue, Yanbing
  organization: Department of Computer Science, University of Pittsburgh, Pittsburgh, PA
– sequence: 2
  givenname: Milos
  surname: Hauskrecht
  fullname: Hauskrecht, Milos
  organization: Department of Computer Science, University of Pittsburgh, Pittsburgh, PA
BackLink https://www.ncbi.nlm.nih.gov/pubmed/28979827$$D View this record in MEDLINE/PubMed
BookMark eNo1jz1PwzAYhD0U0VL6A1iQR5YEv47rj7GKaEEKYgDmyHFeg2k-SpyC-PdEokx30p1O91yQWdd3SMgVsBQgU7eQGqVBAhgljMpSMSMLDlIlDBifk1WMH4xNfq0EZ-dkzrVRRnO1IPnGjeELaYF26EL3RntP88bGGHxwdgx9Rx_7GptIv8P4Touwx2FMnp1tkG4R68q6_SU587aJuDrpkrxu717y-6R42j3kmyI5cC5Eor3xPrMZrq0GMb0x9VpXuuJMe3C81tIJRGaFFg6cEVJAVnnnOJeqmlK-JDd_u4eh_zxiHMs2RIdNYzvsj7EEI5QEySayJbk-VY9Vi3V5GEJrh5_yH5z_ApxfWcU
ContentType Journal Article
DBID NPM
7X8
DOI 10.1137/1.9781611974973.4
DatabaseName PubMed
MEDLINE - Academic
DatabaseTitle PubMed
MEDLINE - Academic
DatabaseTitleList MEDLINE - Academic
PubMed
Database_xml – sequence: 1
  dbid: NPM
  name: PubMed
  url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
  sourceTypes: Index Database
– sequence: 2
  dbid: 7X8
  name: MEDLINE - Academic
  url: https://search.proquest.com/medline
  sourceTypes: Aggregation Database
DeliveryMethod no_fulltext_linktorsrc
Discipline Computer Science
ExternalDocumentID 28979827
Genre Journal Article
GrantInformation_xml – fundername: NIGMS NIH HHS
  grantid: R01 GM088224
– fundername: NLM NIH HHS
  grantid: R01 LM010019
GroupedDBID 7WY
7X2
7XC
88I
8CJ
8FE
8FG
8FH
8FL
8G5
ABJCF
ABUWG
ACGOD
ACIWK
ACPRK
ADBBV
AFKRA
AFRAH
ALMA_UNASSIGNED_HOLDINGS
ARAPS
ATCPS
AZQEC
BBNVY
BENPR
BEZIV
BGLVJ
BHPHI
BPHCQ
CCPQU
CZ9
D1I
D1J
D1K
DWQXO
FRNLG
GNUQQ
GUQSH
HCIFZ
K6-
K60
K6V
K6~
K7-
KB.
KC.
L6V
LK5
LK8
M0C
M0K
M1Q
M2O
M2P
M7P
M7R
M7S
NPM
P62
PATMY
PDBOC
PHGZM
PHGZT
PQBIZ
PQBZA
PQQKQ
PROAC
PTHSS
PYCSY
7X8
ID FETCH-LOGICAL-p2244-8f9ff3a3e5a8140109d58b8b208f1c2d86c4ee0a484c1c946413bfcc2267bd862
IEDL.DBID 7X8
ISSN 2167-0102
IngestDate Fri Jul 11 09:40:13 EDT 2025
Mon Jul 21 06:05:46 EDT 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-p2244-8f9ff3a3e5a8140109d58b8b208f1c2d86c4ee0a484c1c946413bfcc2267bd862
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
OpenAccessLink https://epubs.siam.org/doi/pdf/10.1137/1.9781611974973.4
PMID 28979827
PQID 1947616098
PQPubID 23479
ParticipantIDs proquest_miscellaneous_1947616098
pubmed_primary_28979827
PublicationCentury 2000
PublicationDate 2017-00-00
20170101
PublicationDateYYYYMMDD 2017-01-01
PublicationDate_xml – year: 2017
  text: 2017-00-00
PublicationDecade 2010
PublicationPlace United States
PublicationPlace_xml – name: United States
PublicationTitle Proceedings of the ... SIAM International Conference on Data Mining
PublicationTitleAlternate Proc SIAM Int Conf Data Min
PublicationYear 2017
SSID ssj0001057420
Score 1.7314527
Snippet Annotation of classification data by humans can be a time-consuming and tedious process. Finding ways of reducing the annotation effort is critical for...
SourceID proquest
pubmed
SourceType Aggregation Database
Index Database
StartPage 28
Title Active Learning of Classification Models with Likert-Scale Feedback
URI https://www.ncbi.nlm.nih.gov/pubmed/28979827
https://www.proquest.com/docview/1947616098
Volume 2017
hasFullText
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3NS8MwFA_qPHhxfju_iOC1W5tkTXKSMRwe5hiosFtpvmQo7bTq3-9LlrGTIHgplFIo77287_5-CN1wQ8AqlEmkplCggMdLhKUMboUFdQuTmYCuP-aTiZjN5DQ23Jq4VrnyicFRm1r7HnkPim2ouPNUitvFe-JZo_x0NVJobKIWhVTGr3TxmVj3WCAZYQGZkXh4bw-fFgebGeW9rOvxnnI_R2OS0y77PckMwWbU_u9n7qHdmGbiwdIu9tGGrQ5Qe0XhgOOJPkTDQfB3OMKsvuDa4cCT6TeIgtKwZ0t7a7Bv2OLx3P_jkzyCYi0eQeBTpX49Qs-ju6fhfRJ5FZIFBGyWCCedoyW1_dLjXWWpNH2hhCKpcJkmRuSaWZuWTDCdaclyCHTKaQ2ZGlfwlByjraqu7CnCWkqQnaQszwxTSpZ9kxJNBBSahuZGdtD1SkwF2K0fRpSVrb-aYi2oDjpZyrpYLAE2CigCuRSEn_3h7XO0Q3ykDV2RC9RycGrtJdrW35_z5uMqGARcJ9OHHxEsvNo
linkProvider ProQuest
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Active+Learning+of+Classification+Models+with+Likert-Scale+Feedback&rft.jtitle=Proceedings+of+the+...+SIAM+International+Conference+on+Data+Mining&rft.au=Xue%2C+Yanbing&rft.au=Hauskrecht%2C+Milos&rft.date=2017-01-01&rft.issn=2167-0102&rft.volume=2017&rft.spage=28&rft_id=info:doi/10.1137%2F1.9781611974973.4&rft_id=info%3Apmid%2F28979827&rft_id=info%3Apmid%2F28979827&rft.externalDocID=28979827
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2167-0102&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2167-0102&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2167-0102&client=summon