Nonlinear Feature Construction with Evolved Neural Networks for Classification Problems

Predicting the class membership of a set of patterns represented by points in a multi-dimensional space critically depends on their specific distribution. To improve the classification performance, pattern vectors may be transformed. There is a range of linear methods for feature construction, but these are often limited in their performance. Nonlinear methods are a more recent development in this field, but these pose difficult optimization problems. Evolutionary approaches have been used to optimize both linear and nonlinear functions for feature construction. For nonlinear feature construction, a particular problem is how to encode the function in order to limit the huge search space while preserving enough flexibility to evolve effective solutions. In this paper, we present a new method for generating a nonlinear function for feature construction using multi-layer perceptrons whose weights are shaped by evolution. By pre-defining the architecture of the neural network we can directly influence the computational capacity of the function and the number of features to be constructed. We evaluate the suggested neural feature construction on four commonly used data sets and report an improvement in classification accuracy ranging from 4 to 13 percentage points over the performance on the original pattern set. A version of this report has been published as “Tobias Berka and Helmut A. Mayer: Nonlinear Feature Construction with Evolved Neural Networks for Classification Problems. Proceedings of the First International Conference on Pattern Recognition Applications and Methods (ICPRAM’12), 2012.”