Linear Regression in High Dimension and/or for Correlated Inputs

¹ Université Lille 1 & CNRS & Inria, France
² Institut de Planétologie et d'Astrophysique de Grenoble (IPAG), France

Abstract

Ordinary least square is the common way to estimate linear regression models. When inputs are correlated or when they are too numerous, regression methods using derived inputs directions or shrinkage methods can be efficient alternatives. Methods using derived inputs directions build new uncorrelated variables as linear combination of the initial inputs, whereas shrinkage methods introduce regularization and variable selection by penalizing the usual least square criterion. Both kinds of methods are presented and illustrated thanks to the R software on an astronomical dataset.