Confidence sets for model selection by F -testing

We introduce the notion of variable selection con dence set (VSCS) for linear regression based on F -testing. Our method identi es the most important variables in a principled way that goes beyond simply trusting the single lucky winner based on a model selection criterion. The VSCS extends the usual notion of con dence intervals to the variable selection problem: A VSCS is a set of regression models that contains the true model with a given level of con dence. Although the size of the VSCS properly re ects the model selection uncertainty, without speci c assumptions on the true model, the VSCS is typically rather large (unless the number of predictors is small). As a solution, we advocate special attention to the set of lower boundary models (LBMs), which are the most parsimonious models that are not statistically signi cantly inferior to the full model at a given con dence level. Based on the LBMs, variable importance and measures of co-appearance importance of predictors can be naturally de ned.