Generalized SMO Algorithm for SVM-Based Multitask Learning

Exploiting additional information to improve traditional inductive learning is an active research area in machine learning. In many supervised-learning applications, training data can be naturally separated into several groups, and incorporating this group information into learning may improve gener...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems Vol. 23; no. 6; pp. 997 - 1003
Main Authors: Feng Cai, Cherkassky, V.
Format: Journal Article
Language:English
Published: New York, NY IEEE 01.06.2012
Institute of Electrical and Electronics Engineers
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Algorithms
Applied sciences
Artificial intelligence
Classification
Computational efficiency
Computer science; control theory; systems
Data processing. List processing. Character string processing
Exact sciences and technology
Formulations
Learning
Learning and adaptive systems
learning with structured data
Machine learning
Memory organisation. Data processing
multitask learning
Optimization
quadratic optimization
sequential minimal optimization
Software
Studies
support vector machine (SVM)
Support vector machines
Training
Training data
Vectors
Zirconium
{\rm SVM}{+}
Sample size
support vector machine (SVM)
sequential minimal optimization
learning with structured data
SVM
Quadratic programming
multitask learning
quadratic optimization
Inductive learning
Optimization
Active system
Supervised learning
Sequential method
Internal structure
Concurrent task
Classification
Vector support machine
Probability learning
Data structure
Inductive logic programming
Learning algorithm
Artificial intelligence
Mathematical programming
ISSN:2162-237X, 2162-2388
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Exploiting additional information to improve traditional inductive learning is an active research area in machine learning. In many supervised-learning applications, training data can be naturally separated into several groups, and incorporating this group information into learning may improve generalization. Recently, Vapnik proposed a general approach to formalizing such problems, known as "learning with structured data" and its support vector machine (SVM) based optimization formulation called SVM+. Liang and Cherkassky showed the connection between SVM+ and multitask learning (MTL) approaches in machine learning, and proposed an SVM-based formulation for MTL called SVM+MTL for classification. Training the SVM+MTL classifier requires the solution of a large quadratic programming optimization problem which scales as O(n 3 ) with sample size n. So there is a need to develop computationally efficient algorithms for implementing SVM+MTL. This brief generalizes Platt's sequential minimal optimization (SMO) algorithm to the SVM+MTL setting. Empirical results show that, for typical SVM+MTL problems, the proposed generalized SMO achieves over 100 times speed-up, in comparison with general-purpose optimization routines.
Bibliography:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
content type line 23
ObjectType-Article-1
ObjectType-Feature-2
ISSN:2162-237X
2162-2388
DOI:10.1109/TNNLS.2012.2187307