Prem S. Puri Memorial Lecture
Penalized Linear Unbiased Selection Via Non-Convex Minimization
Professor Cun-Hui Zhang
Department of Statistics, Rutgers University
Start Date and Time: Thu, 5 Apr 2007, 4:30 PM
End Date and Time: Thu, 5 Apr 2007, 6:00 PM
Venue: MATH 175
Abstract:
We offer fast, continuous, nearly unbiased, and accurate penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO interfere with variable selection. Subset selection is unbiased but computationally costly. We propose a minimax concave penalty (MCP) which provides the minimum non-convexity of the penalized loss given the level of bias. We introduce a PLUS algorithm which computes many local minimizers of the penalized loss in a unique main branch of such solutions for non-convex penalized loss functions. The output of the PLUS is a continuous piecewise linear path encompassing from the origin to an optimal solution for zero penalty. We propose to choose the sparsest solution within the PLUS path for a given penalty level. We provide a necessary and sufficient condition for the continuity of the penalized LSE under general sub-square penalties, and a sufficient and nearly necessary condition for the validity of the Stein formula for unbiased estimation of the risk. We develop the degrees of freedom and Cp-type risk estimates for general penalized LSE, including the LASSO estimator, and prove their unbiasedness. We prove that the proposed MC+ methodology has high probability of correct selection under much weaker conditions compared with existing results for the LASSO for large n and p, including the case of p >> n.