DCA based approaches for bi-level variable selection and application for estimate multiple sparse covariance matrices - Le Thi, Hoai An; Phan, Duy Nhat; Pham Dinh, Tao (Neurocomputing (Amsterdam), 2021)
•Two new models via new regularizations for bi-level variable selection are considered.•The first uses the mixed zero norm and the second is the combined ℓ0+ℓq,0 norm.•All the resulting optimization problems are formulated as DC programs.•DCA are investigated via suitable DC decompositions for these DC programs.•The algorithms are applied on estimation of multiple sparse covariance matrices.A careful empirical experiment on both simulated and real datasets are performed. Variable selection plays an important role in analyzing high dimensional data and is a fundamental problem in machine learning. When the data possesses certain group structures in which individual variables are also meaningful scientifically, we are naturally interested in selecting important groups as well as important variables within the selected groups. This is referred to as the bi-level variable selection which is much more complex than the selection of individual variables. In recent years, research on the topic of variable selection is very active, but the majority of the work is focused on the individual variable selection. There is therefore a need to further develop more effective approaches for bi-level variable selection. Since DC (Difference of Convex functions) programming and DCA (DC Algorithm), powerful tools in nonconvex programming framework, have been successfully investigated for individual variable selection, we believe that they could be efficiently exploited for the more difficult bi-level variable selection task. In that direction, we investigate in this work DC approximations of the mixed zero norm (ℓ0,0) and the combined norm (ℓ0+ℓq,0). The resulting approximate problems are then formulated as DC programs for which DCA based algorithms are introduced. As an application, these DCA schemes are developed for estimating multiple sparse covariance matrices sharing some common structures such as the locations or weights of non-zero elements. The experimental results on both simulated and real datasets indicate the efficiency of our algorithms.