Sparse Covariance Thresholding for High-Dimensional Variable Selection with the Lasso

Ms. Jessie Jeng and Mr. John Daye

An integral topic in statistical research is variable selection for high-dimensional 
applications, where the number of predictors p is large relative to the number of 
observations n. High-dimensional data often come from observational studies, in which 
predictors are considered to be random. One of the most popular methods for variable 
selection is the lasso, which has been successful in many applications. However, under 
the 'large p small n' scenario, the lasso is not very satisfactory due to excessive 
variability and rank deficiency of the sample covariance matrix. These limitations can be 
mitigated when the covariance matrix is known to be sparse. Covariance sparsity is a 
natural assumption in many applications such as gene microarray analysis, image 
processing, etc., in which a large number of predictors are independent or weakly 
correlated with each other. In this paper, we propose to apply generalized 
covariance-thresholding to stabilize and improve the performances of the lasso under the 
covariance sparsity assumption. We call this procedure the covariance-thresholded lasso. 
We establish consistency results that relate the sparsity of the covariance matrix with 
variable selection and modify the LARS algorithm for our method. Finite sample 
performances are examined using simulation and real-data examples. Results indicate that 
our method can improve upon the lasso, adaptive lasso, and elastic net in prediction 
accuracy and variable selection for high-dimensional applications, particularly when n << 
p.

This is a joint work with Michael Y. Zhu.