The “bottleneck” behaviours in linear feedforward neural network classifiers and their breakthrough

The classification mechanisms of linear feedforward neural network classifiers (FNNC), whose hidden layer performs the Fisher linear transformation of the input patterns, under the supervision of outer-supervised signals are investigated. The “bottleneck” behaviours in linear FNNCs are observed and analyzed. In addition, the structure stabilities of the linear FNNCs are also discussed. It is pointed out that the key point to break through the “bottleneck” behaviours for linear FNNCs is to change linear hidden neurons into nonlinear hidden ones. Finally, the experimental results, taking the parity 3 problem as example, are given.