A novel multi-objective forest optimization algorithm for wrapper feature selection

Abstract Feature selection is one of the important techniques of dimensionality reduction in data preprocessing because datasets generally have redundant and irrelevant features that adversely affect the performance and complexity of classification models. Feature selection has two main objectives, i.e., reducing the number of features and increasing classification performance due to its inherent nature. In this paper, we propose a multi-objective feature selection algorithm based on forest optimization algorithm (FOA) using the archive, grid, and region-based selection concepts. For this purpose, two versions of the proposed algorithm are developed using continuous and binary representations. The performance of the proposed algorithms is investigated on nine UCI datasets and two microarray datasets. Next, the obtained results are compared with seven traditional single-objective and five multi-objective methods. Based on the results, both proposed algorithms have reached the same performance or even outperformed the single-objective methods. Compared with other multi-objective algorithms, MOFOA with continuous representation has managed to reduce the classification error in most cases by selecting less number of features than other methods.