Alternating DCA for reduced-rank multitask linear regression with covariance matrix estimation

We study a challenging problem in machine learning that is the reduced-rank multitask linear regression with covariance matrix estimation. The objective is to build a linear relationship between multiple output variables and input variables of a multitask learning process, taking into account the ge...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Annals of mathematics and artificial intelligence Ročník 90; číslo 7-9; s. 809 - 829
Hlavní autoři: Le Thi, Hoai An, Ho, Vinh Thanh
Médium: Journal Article
Jazyk:angličtina
Vydáno: Cham Springer International Publishing 01.09.2022
Springer
Springer Nature B.V
Springer Verlag
Témata:
Algorithms
Artificial Intelligence
Complex Systems
Computer Science
Convexity
Covariance matrix
Critical point
Machine learning
Mathematics
Regression analysis
Regression models
DCA
62J05
Reduced-rank multitask linear regression
DC programming
90C90
90C26
Partial DC program
Covariance matrix estimation
Alternating DCA
Partial DC programming
ISSN:1012-2443, 1573-7470
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We study a challenging problem in machine learning that is the reduced-rank multitask linear regression with covariance matrix estimation. The objective is to build a linear relationship between multiple output variables and input variables of a multitask learning process, taking into account the general covariance structure for the errors of the regression model in one hand, and reduced-rank regression model in another hand. The problem is formulated as minimizing a nonconvex function in two joint matrix variables ( X , Θ ) under the low-rank constraint on X and positive definiteness constraint on Θ . It has a double difficulty due to the non-convexity of the objective function as well as the low-rank constraint. We investigate a nonconvex, nonsmooth optimization approach based on DC (Difference of Convex functions) programming and DCA (DC Algorithm) for this hard problem. A penalty reformulation is considered which takes the form of a partial DC program. An alternating DCA and its inexact version are developed, both algorithms converge to a weak critical point of the considered problem. Numerical experiments are performed on several synthetic and benchmark real multitask linear regression datasets. The numerical results show the performance of the proposed algorithms and their superiority compared with three classical alternating/joint methods.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1012-2443
1573-7470
DOI:10.1007/s10472-021-09732-8