Safe RuleFit: Learning Optimal Sparse Rule Model by Meta Safe Screening

We consider the problem of learning a sparse rule model , a prediction model in the form of a sparse linear combination of rules, where a rule is an indicator function defined over a hyper-rectangle in the input space. Since the number of all possible such rules is extremely large, it has been compu...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE Transactions on Pattern Analysis and Machine Intelligence Ročník 45; číslo 2; s. 2330 - 2343
Hlavní autoři: Kato, Hiroki, Hanada, Hiroyuki, Takeuchi, Ichiro
Médium: Journal Article
Jazyk:angličtina
Vydáno: United States IEEE 01.02.2023
Institute of Electrical and Electronics Engineers (IEEE)
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Analytical models
combinatorial algorithms
convex programming
Dictionaries
knowledge representation formalisms and methods
Learning
Machine learning
Numerical models
Pattern analysis
Prediction models
Predictive models
Random forests
Regression tree analysis
Regularization
Screening
ISSN:0162-8828, 1939-3539, 1939-3539, 2160-9292
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We consider the problem of learning a sparse rule model , a prediction model in the form of a sparse linear combination of rules, where a rule is an indicator function defined over a hyper-rectangle in the input space. Since the number of all possible such rules is extremely large, it has been computationally intractable to select the optimal set of active rules. In this paper, to solve this difficulty for learning the optimal sparse rule model, we propose Safe RuleFit (SRF) . Our basic idea is to develop meta safe screening (mSS) , which is a non-trivial extension of well-known safe screening (SS) techniques. While SS is used for screening out one feature, mSS can be used for screening out multiple features by exploiting the inclusion-relations of hyper-rectangles in the input space. SRF provides a general framework for fitting sparse rule models for regression and classification, and it can be extended to handle more general sparse regularizations such as group regularization. We demonstrate the advantages of SRF through intensive numerical experiments.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:0162-8828
1939-3539
1939-3539
2160-9292
DOI:10.1109/TPAMI.2022.3167993