TSVMPath: Fast Regularization Parameter Tuning Algorithm for Twin Support Vector Machine

Twin support vector machine (TSVM) has attracted much attention in the field of machine learning with good generalization ability and computational performance. However, the conventional grid search method is very time-consuming to obtain the optimal regularization parameter. To address this problem...

Full description

Saved in:
Bibliographic Details
Published in:Neural processing letters Vol. 54; no. 6; pp. 5457 - 5482
Main Authors: Zhou, Kanglei, Zhang, Qiyang, Li, Juntao
Format: Journal Article
Language:English
Published: New York Springer US 01.12.2022
Springer Nature B.V
Subjects:
Accuracy
Algorithms
Artificial Intelligence
Classification
Complex Systems
Computational Intelligence
Computer Science
Design
Efficiency
Lagrange multiplier
Machine learning
Optimization
Optimization models
Parameters
Quadratic programming
Regularization
Search methods
Support vector machines
Tuning
Statistical machine learning
Regularized solution path
Parameter tuning algorithm
Twin support vector machine
ISSN:1370-4621, 1573-773X
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Twin support vector machine (TSVM) has attracted much attention in the field of machine learning with good generalization ability and computational performance. However, the conventional grid search method is very time-consuming to obtain the optimal regularization parameter. To address this problem, we develop a novel fast regularization parameter tuning algorithm for TSVM, named TSVMPath. After transforming the models of two sub-optimization problems, we divide the two classes of samples into different sets. Lagrangian multipliers are then proved to be piecewise linear concerning the corresponding regularization parameters, greatly extending the search space of the solution. By proving that the Lagrangian multipliers of two sub-optimization models are 1 when the regularization parameters approach infinity, we design a simple yet effective initialization. As a result, the entirely regularized solution path can be obtained without solving quadratic programming problems. Four types of events are finally defined to update the solution path. Experiments on 8 UCI datasets show that the prediction accuracy of TSVMPath is superior to the best competing methods, with up to four orders of magnitude speed-up for the computational overhead compared with the grid search method.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1370-4621
1573-773X
DOI:10.1007/s11063-022-10870-1