Prem S. Puri Memorial Lecture

Penalized Linear Unbiased Selection Via Non-Convex Minimization

Professor Cun-Hui Zhang
Department of Statistics, Rutgers University

Start Date and Time: Thu, 5 Apr 2007, 4:30 PM

End Date and Time: Thu, 5 Apr 2007, 6:00 PM

Venue: MATH 175


We offer fast, continuous, nearly unbiased, and accurate penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO interfere with variable selection. Subset selection is unbiased but computationally costly. We propose a minimax concave penalty (MCP) which provides the minimum non-convexity of the penalized loss given the level of bias. We introduce a PLUS algorithm which computes many local minimizers of the penalized loss in a unique main branch of such solutions for non-convex penalized loss functions. The output of the PLUS is a continuous piecewise linear path encompassing from the origin to an optimal solution for zero penalty. We propose to choose the sparsest solution within the PLUS path for a given penalty level. We provide a necessary and sufficient condition for the continuity of the penalized LSE under general sub-square penalties, and a sufficient and nearly necessary condition for the validity of the Stein formula for unbiased estimation of the risk. We develop the degrees of freedom and Cp-type risk estimates for general penalized LSE, including the LASSO estimator, and prove their unbiasedness. We prove that the proposed MC+ methodology has high probability of correct selection under much weaker conditions compared with existing results for the LASSO for large n and p, including the case of p >> n

Simulation results overwhelmingly support our claim of superior variable selection properties and demonstrate the computational efficiency of the proposed MC+. Extensions to generalized linear models are discussed.

Last Updated: Aug 31, 2017 11:22 AM

Purdue Department of Statistics, 250 N. University St, West Lafayette, IN 47907

Phone: (765) 494-6030, Fax: (765) 494-0558

© 2015 Purdue University | An equal access/equal opportunity university | Copyright Complaints

Trouble with this page? Disability-related accessibility issue? Please contact the College of Science Webmaster.