A new large-scale learning algorithm for generalized additive models

Additive model plays an important role in machine learning due to its flexibility and interpretability in the prediction function. However, solving large-scale additive models is a challenging task due to several difficulties. Until now, scaling up additive models is still an open problem. To addres...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Machine learning Ročník 112; číslo 9; s. 3077 - 3104
Hlavní autoři: Gu, Bin, Zhang, Chenkang, Huo, Zhouyuan, Huang, Heng
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.09.2023
Springer Nature B.V
Témata:
Algorithms
Artificial Intelligence
Basis functions
Computer Science
Computing time
Control
Convergence
Machine Learning
Mechatronics
Natural Language Processing (NLP)
Optimization
Robotics
Simulation and Modeling
Wrapper algorithm
Doubly stochastic gradient
Additive model
ISSN:0885-6125, 1573-0565
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Additive model plays an important role in machine learning due to its flexibility and interpretability in the prediction function. However, solving large-scale additive models is a challenging task due to several difficulties. Until now, scaling up additive models is still an open problem. To address this challenging problem, in this paper, we propose a new doubly stochastic optimization algorithm for solving the generalized additive models (DSGAM). We first propose a generalized formulation of additive models without the orthogonal hypothesis on the basis function. After that, we propose a wrapper algorithm to optimize the generalized additive models. Importantly, we introduce a doubly stochastic gradient algorithm (DSG) to solve an inner subproblem in the wrapper algorithm, which can scale well in sample size and dimensionality simultaneously. Finally, we prove the fast convergence rate of our DSGAM algorithm. The experimental results on various large-scale benchmark datasets not only confirm the fast convergence of our DSGAM algorithm, but also show a huge reduction of computational time compared with existing algorithms, while retaining the similar generalization performance.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0885-6125
1573-0565
DOI:10.1007/s10994-023-06339-4