Feature Selection Using Probabilistic Neural Networks

Selection of input variables (features) is a key stage in building predictive models. As exhaustive evaluation of potential feature sets using full non-linear models is impractical, it is common practice to use simple fast-evaluating models and heuristic selection strategies. This paper discusses a fast, efficient, and powerful non-linear input selection procedure using a combination of probabilistic neural networks and repeated bitwise gradient descent with resampling. The algorithm is compared with forward selection, backward selection and genetic algorithms using a selection of real-world data sets. The algorithm has comparative performance and greatly reduced execution time with respect to these alternative approaches.