Large-Scale Feedforward Neural Network Optimization by a Self-Adaptive Strategy and Parameter Based Particle Swarm Optimization

Feedforward neural network (FNN) is one of the most widely used and fastest-developed artificial neural networks. Much evolutionary computation (EC) methods have been used to optimize the weights of FNN. However, as the dimension of datasets increases, the number of weights also increases dramatically. On high-dimensional datasets, if EC methods are used directly to optimize the weights of FNN, it is impossible to obtain the optimal weights of the FNN by EC methods in an acceptable time. Feature selection is a method that can effectively reduce the computational complexity of FNN by reducing irrelevant and redundant features. It may be practical to optimize the FNN by EC methods if we first employ the feature selection for the large-scale datasets. In this paper, we present a self-adaptive parameter and strategy-based particle swarm optimization (SPS-PSO) algorithm to optimize FNN with feature selection. First, we propose an optimization model for FNN by transforming the designing of FNN into a weights optimization problem. Simultaneously, we present a feature selection optimization model. Second, we present an SPS-PSO algorithm. In this algorithm, we use the parameter and strategy self-adaptive mechanism. In addition, five candidate solution generating strategies (CSGS) are used. The experiments are divided into two groups. In the first group, SPS-PSO and three other EC methods are used to directly optimize the weights of FNN on eight datasets without any modification. In the second group, we first employ SPS-PSO-based feature selection on the original datasets and obtain eight relatively smaller datasets with the ${k}$ -nearest neighbor (KNN) which is used as the evaluation function for saving time. Then, we use the new datasets as the inputs for FNN. We optimize the weights of FNN again by SPS-PSO and three other EC methods to investigate whether we can get similar or even better classification accuracy by comparing the results with that of the first group. The experimental results show that SPS-PSO has the advantage in optimizing the weights of FNN compared with the other EC methods. Meanwhile, the SPS-PSO-based feature selection can reduce the solution size and computational complexity while ensuring the classification accuracy when it is used to preprocess the datasets for FNN. In this method, a solution with an originally higher than 700 000 dimensions can be even reduced to hundreds of dimensions.