On the overfitting of the five-layered bottleneck network

In autoassociative learning for the bottleneck neural network, the problem of overfitting is pointed out. This overfitting is pathological in the sense that it does not disappear even if the sample size goes to infinity. However, it is not observed in the real learning process. Thus we study the basin of the overfitting solution. First, the existence of overfitting is confirmed. Then it is shown that the basin of the overfitting solution is small compared with the normal solution.

[1]  Ken‐Ichi Funahashi,et al.  Approximate realization of identity mappings by three-layer neural networks , 1990 .

[2]  T. McAvoy,et al.  Nonlinear principal component analysis—Based on principal curves and neural networks , 1996 .

[3]  Naohiro Fukumura,et al.  A computational model for recognizing objects and planning hand shapes in grasping movements , 1995, Neural Networks.

[4]  Kazuyuki Hiraoka,et al.  Data Fitting by a Broad Class of Surfaces Using Multiplied Bottleneck Networks. , 1998 .

[5]  Isamu Kajitani,et al.  Evolvable Hardware for Generalized Neural Networks , 1997, IJCAI.

[6]  Shufeng Tan,et al.  Reducing data dimensionality through optimizing neural network inputs , 1995 .

[7]  Paolo F. Fantoni,et al.  Multiple-Failure Signal Validation in Nuclear Power Plants Using Artificial Neural Networks , 1996 .

[8]  M. Kramer.  Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[9]  Shigeki Nakauchi,et al.  Reconstruction of Munsell color space by a five-layer neural network , 1992 .

[10]  D. Ruppert,et al.  Measurement Error in Nonlinear Models , 1995 .

[11]  R. Tibshirani,et al.  Adaptive Principal Surfaces , 1994 .

[12]  E. C. Malthouse,et al.  Limitations of nonlinear PCA as performed with generic neural networks , 1998, IEEE Trans. Neural Networks.

[13]  I. Noda.  Learning method by overload , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[14]  Kazuyuki Hiraoka,et al.  Recalling of Many-Valued Functions by Successive Iteration on Bottleneck Networks , 1998, ICONIP.