New Scalable and Efficient Online Pairwise Learning Algorithm

Pairwise learning is an important machine-learning topic with many practical applications. An online algorithm is the first choice for processing streaming data and is preferred for handling large-scale pairwise learning problems. However, existing online pairwise learning algorithms are not scalabl...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems Vol. 35; no. 12; pp. 17099 - 17110
Main Authors: Gu, Bin, Bao, Runxue, Zhang, Chenkang, Huang, Heng
Format: Journal Article
Language:English
Published: United States IEEE 01.12.2024
Subjects:
Doubly stochastic gradient algorithm
Electronic learning
Heuristic algorithms
Machine learning
Measurement
online learning
pairwise learning
Scalability
statistical accuracy
Statistics
Stochastic processes
Training
ISSN:2162-237X, 2162-2388, 2162-2388
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Pairwise learning is an important machine-learning topic with many practical applications. An online algorithm is the first choice for processing streaming data and is preferred for handling large-scale pairwise learning problems. However, existing online pairwise learning algorithms are not scalable and efficient enough for large-scale high-dimensional data, because they were designed based on singly stochastic gradients. To address this challenging problem, in this article, we propose a dynamic doubly stochastic gradient algorithm (D2SG) for online pairwise learning. Especially, only the time and space complexities of <inline-formula> <tex-math notation="LaTeX">\mathcal {O} (d) </tex-math></inline-formula> are needed for incorporating a new sample, where <inline-formula> <tex-math notation="LaTeX">d </tex-math></inline-formula> is the dimensionality of data. This means that our D2SG is much faster and more scalable than the existing online pairwise learning algorithms while the statistical accuracy can be guaranteed through our rigorous theoretical analysis under standard assumptions. The experimental results on a variety of real-world datasets not only confirm the theoretical result of our new D2SG algorithm, but also show that D2SG has better efficiency and scalability than the existing online pairwise learning algorithms.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:2162-237X
2162-2388
2162-2388
DOI:10.1109/TNNLS.2023.3299756