Another look at distance-weighted discrimination

Distance-weighted discrimination (DWD) is a modern margin-based classifier with an interesting geometric motivation. It was proposed as a competitor to the support vector machine (SVM). Despite many recent references on DWD, DWD is far less popular than the SVM, mainly because of computational and t...

Full description

Saved in:
Bibliographic Details
Published in:Journal of the Royal Statistical Society. Series B, Statistical methodology Vol. 80; no. 1; pp. 177 - 198
Main Authors: Wang, Boxiang, Zou, Hui
Format: Journal Article
Language:English
Published: Oxford John Wiley & Sons Ltd 01.01.2018
Oxford University Press
Subjects:
Algorithms
Bayes risk consistency
Bayesian analysis
Classification
Classifiers
Computation
data collection
Discrimination
Distance‐weighted discrimination
equations
Hilbert space
Kernel learning
Learning
Learning theories
Learning theory
Machine learning
Majorization–minimization principle
Motivation
Novels
Regression analysis
risk
Second‐order cone programming
Statistical methods
Statistics
Support vector machines
ISSN:1369-7412, 1467-9868
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Distance-weighted discrimination (DWD) is a modern margin-based classifier with an interesting geometric motivation. It was proposed as a competitor to the support vector machine (SVM). Despite many recent references on DWD, DWD is far less popular than the SVM, mainly because of computational and theoretical reasons. We greatly advance the current DWD methodology and its learning theory. We propose a novel thrifty algorithm for solving standard DWD and generalized DWD, and our algorithm can be several hundred times faster than the existing state of the art algorithm based on second-order cone programming. In addition, we exploit the new algorithm to design an efficient scheme to tune generalized DWD. Furthermore, we formulate a natural kernel DWD approach in a reproducing kernel Hubert space and then establish the Bayes risk consistency of the kernel DWD by using a universal kernel such as the Gaussian kernel. This result solves an open theoretical problem in the DWD literature. A comparison study on 16 benchmark data sets shows that data-driven generalized DWD consistently delivers higher classification accuracy with less computation time than the SVM.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:1369-7412
1467-9868
DOI:10.1111/rssb.12244