How to Evaluate Foreground Maps

The output of many algorithms in computer-vision is either non-binary maps or binary maps (e.g., salient object detection and object segmentation). Several measures have been suggested to evaluate the accuracy of these foreground maps. In this paper, we show that the most commonly-used measures for...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2014 IEEE Conference on Computer Vision and Pattern Recognition s. 248 - 255
Hlavní autori: Margolin, Ran, Zelnik-Manor, Lihi, Tal, Ayellet
Médium: Konferenčný príspevok.. Journal Article
Jazyk:English
Vydavateľské údaje: IEEE 01.06.2014
Predmet:
Accuracy
Adequacy
Algorithms
Area measurement
Computer vision
Conferences
Current measurement
Equations
evaluation
foreground extraction
Interpolation
meta-measure
Object detection
Pascal (programming language)
Pattern recognition
saliency
Segmentation
Vectors
Volatile organic compounds
ISSN:1063-6919, 1063-6919
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Abstract The output of many algorithms in computer-vision is either non-binary maps or binary maps (e.g., salient object detection and object segmentation). Several measures have been suggested to evaluate the accuracy of these foreground maps. In this paper, we show that the most commonly-used measures for evaluating both non-binary maps and binary maps do not always provide a reliable evaluation. This includes the Area-Under-the-Curve measure, the Average-Precision measure, the F β -measure, and the evaluation measure of the PASCAL VOC segmentation challenge. We start by identifying three causes of inaccurate evaluation. We then propose a new measure that amends these flaws. An appealing property of our measure is being an intuitive generalization of the F β -measure. Finally we propose four meta-measures to compare the adequacy of evaluation measures. We show via experiments that our novel measure is preferable.
AbstractList The output of many algorithms in computer-vision is either non-binary maps or binary maps (e.g., salient object detection and object segmentation). Several measures have been suggested to evaluate the accuracy of these foreground maps. In this paper, we show that the most commonly-used measures for evaluating both non-binary maps and binary maps do not always provide a reliable evaluation. This includes the Area-Under-the-Curve measure, the Average-Precision measure, the F-measure, and the evaluation measure of the PASCAL VOC segmentation challenge. We start by identifying three causes of inaccurate evaluation. We then propose a new measure that amends these flaws. An appealing property of our measure is being an intuitive generalization of the F-measure. Finally we propose four meta-measures to compare the adequacy of evaluation measures. We show via experiments that our novel measure is preferable.
The output of many algorithms in computer-vision is either non-binary maps or binary maps (e.g., salient object detection and object segmentation). Several measures have been suggested to evaluate the accuracy of these foreground maps. In this paper, we show that the most commonly-used measures for evaluating both non-binary maps and binary maps do not always provide a reliable evaluation. This includes the Area-Under-the-Curve measure, the Average-Precision measure, the F β -measure, and the evaluation measure of the PASCAL VOC segmentation challenge. We start by identifying three causes of inaccurate evaluation. We then propose a new measure that amends these flaws. An appealing property of our measure is being an intuitive generalization of the F β -measure. Finally we propose four meta-measures to compare the adequacy of evaluation measures. We show via experiments that our novel measure is preferable.
Author Zelnik-Manor, Lihi
Tal, Ayellet
Margolin, Ran
Author_xml – sequence: 1
  givenname: Ran
  surname: Margolin
  fullname: Margolin, Ran
  email: margolin@tx.technion.ac.il
  organization: Technion - Israel Inst. of Technol., Haifa, Israel
– sequence: 2
  givenname: Lihi
  surname: Zelnik-Manor
  fullname: Zelnik-Manor, Lihi
  email: lihi@ee.technion.ac.il
  organization: Technion - Israel Inst. of Technol., Haifa, Israel
– sequence: 3
  givenname: Ayellet
  surname: Tal
  fullname: Tal, Ayellet
  email: ayellet@ee.technion.ac.il
  organization: Technion - Israel Inst. of Technol., Haifa, Israel
BookMark eNpNjE1Lw0AURUepYK3duXNhlm5S38t85S2ltFaoKKJuw5tkIoE0UzOJ4r9XqAtX93A43DMx6ULnhbhAWCAC3Szfnp4XGaBaSDoSc7I5KkukEXN9LKYIRqaGkCb_-FTMY2wcZMYapaWZiqtN-EqGkKw-uR158Mk69P69D2NXJQ-8j-fipOY2-vnfzsTrevWy3KTbx7v75e02bTLIh9TlirWykDNXSoMra6KqZFRaa6d-HWR1SY6tsTVoUE57y1VZa1YG2KGcievD774PH6OPQ7FrYunbljsfxligsZZQEerf9PKQNt77Yt83O-6_C0NASkr5A-_lTrs
CODEN IEEPAD
ContentType Conference Proceeding
Journal Article
DBID 6IE
6IH
CBEJK
RIE
RIO
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
DOI 10.1109/CVPR.2014.39
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP) 1998-present
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle Technology Research Database
Computer and Information Systems Abstracts – Academic
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
DatabaseTitleList Technology Research Database
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Applied Sciences
Computer Science
EISBN 9781479951185
1479951188
EISSN 1063-6919
EndPage 255
ExternalDocumentID 6909433
Genre orig-research
GroupedDBID 23M
29F
29O
6IE
6IH
6IK
ABDPE
ACGFS
ALMA_UNASSIGNED_HOLDINGS
CBEJK
IPLJI
M43
RIE
RIO
RNS
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
ID FETCH-LOGICAL-i208t-b84a54708aad450bcf99dca14555b4ad402fc9ba767f0504b5e7adcf5a460ab13
IEDL.DBID RIE
ISICitedReferencesCount 881
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000361555600032&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 1063-6919
IngestDate Fri Sep 05 09:26:03 EDT 2025
Wed Aug 27 04:29:57 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i208t-b84a54708aad450bcf99dca14555b4ad402fc9ba767f0504b5e7adcf5a460ab13
Notes ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Conference-1
ObjectType-Feature-3
content type line 23
SourceType-Conference Papers & Proceedings-2
PQID 1677914915
PQPubID 23500
PageCount 8
ParticipantIDs ieee_primary_6909433
proquest_miscellaneous_1677914915
PublicationCentury 2000
PublicationDate 20140601
PublicationDateYYYYMMDD 2014-06-01
PublicationDate_xml – month: 06
  year: 2014
  text: 20140601
  day: 01
PublicationDecade 2010
PublicationTitle 2014 IEEE Conference on Computer Vision and Pattern Recognition
PublicationTitleAbbrev CVPR
PublicationYear 2014
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssib026764536
ssj0023720
ssj0003211698
Score 2.5228257
Snippet The output of many algorithms in computer-vision is either non-binary maps or binary maps (e.g., salient object detection and object segmentation). Several...
SourceID proquest
ieee
SourceType Aggregation Database
Publisher
StartPage 248
SubjectTerms Accuracy
Adequacy
Algorithms
Area measurement
Computer vision
Conferences
Current measurement
Equations
evaluation
foreground extraction
Interpolation
meta-measure
Object detection
Pascal (programming language)
Pattern recognition
saliency
Segmentation
Vectors
Volatile organic compounds
Title How to Evaluate Foreground Maps
URI https://ieeexplore.ieee.org/document/6909433
https://www.proquest.com/docview/1677914915
WOSCitedRecordID wos000361555600032&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELbaioGpQIsozyAxktaJX_FcteoAVYWg6hb5FalLUrUp_H1s58EAC1t0UiTbZ_vufHffB8BTYqTVsopDyhMU4kioUGDBQyaQVTiWkfYIfOsXtlwmmw1fdcBz2wtjjPHFZ2bsPn0uXxfq6J7KJjaS4xihLugyRqterWbvxJRRTCrubn8LIxvZUN5mFGLHxuIznxTZcUW8LYLnk-l69eaKvPDYUYZ7kpVfN7M3N_P-_wZ6BoY_fXvBqrVI56Bj8gvQrx3NoD7GBytquBwa2QA8LIqvoCyCWYX-bQLH2elaPnIdvIrdYQg-5rP36SKsyRPCbQyTMpQJFgQzmAihMYFSZZxrJRwuOZHYymCcKS4FoyyDBGJJDBNaZURgCoWM0CXo5UVurkAgpZaEKGSdDYcXKLg1egYaLRlU1j3SIzBwS5DuKnyMtJ79CDw2a5jaPesSESI3xfGQRpQxbkOziFz__esNOHX6qEqybkGv3B_NHThRn-X2sL_3iv8Ge3io1Q
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT8JAEJ4gmugJFYz4oiYeLWzbfXTPRIIRCDFIuDX7asKlJRT077tbCh704q2ZpMl2Z3ZnpjPzfQBPsZFWyyr0KY8jHwdC-QIL7jMRWYVjGegSgW8-YpNJvFjwaQ2eD7Mwxpiy-cx03WNZy9e52rpfZT2byXEcRUdw7JizqmmtvfWElFFMduzd5T0c2dyG8kNNIXR8LGXtk0Z2ZQE_tMHzXn8-fXdtXrjrSMNLmpVfd3PpcAaN_y31HFo_k3ve9OCTLqBmsktoVKGmVx3kwor2bA57WRM6w_zL2-Teyw7_23iOtdMNfWTaG4tV0YKPwcusP_Qr-gR_GaJ448sYC4IZioXQmCCpUs61Eg6ZnEhsZShMFZeCUZYigrAkhgmtUiIwRUIG0RXUszwz1-BJqSUhKrLhhkMMFNy6PYOMlgwpGyDpNjTdFiSrHUJGUn19Gx73e5hYq3WlCJGZfFskAWWM2-QsIDd_v9qB0-FsPEpGr5O3Wzhzutk1aN1BfbPemns4UZ-bZbF-KI3gG3sJrB4
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2014+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition&rft.atitle=How+to+Evaluate+Foreground+Maps&rft.au=Margolin%2C+Ran&rft.au=Zelnik-Manor%2C+Lihi&rft.au=Tal%2C+Ayellet&rft.date=2014-06-01&rft.pub=IEEE&rft.issn=1063-6919&rft.eissn=1063-6919&rft.spage=248&rft.epage=255&rft_id=info:doi/10.1109%2FCVPR.2014.39&rft.externalDocID=6909433
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1063-6919&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1063-6919&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1063-6919&client=summon