Avoiding overfitting in multilayer perceptrons with feeling-of-knowing using self-organizing maps.

Overfitting in multilayer perceptron (MLP) training is a serious problem. The purpose of this study is to avoid overfitting in on-line learning. To overcome the overfitting problem, we have investigated feeling-of-knowing (FOK) using self-organizing maps (SOMs). We propose MLPs with FOK using the SOMs method to overcome the overfitting problem. In this method, the learning process advances according to the degree of FOK calculated using SOMs. The mean square error obtained for the test set using the proposed method is significantly less than that in a conventional MLP method. Consequently, the proposed method avoids overfitting.

[1]  Eric O. Postma,et al.  Avoiding Overfitting with BP-SOM , 1997, IJCAI.

[2]  A. J. M. M. Weijters.  The BP-SOM architecture and learning rule , 2006, Neural Processing Letters.

[3]  Kenichi Ohki,et al.  Neural Correlates for Feeling-of-Knowing An fMRI Parametric Analysis , 2002, Neuron.

[4]  J. Hart,et al.  Memory and the feeling-of-knowing experience. , 1965, Journal of educational psychology.

[5]  J. Metcalfe.  Feeling of knowing in memory and problem solving. , 1986 .

[6]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[7]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[8]  Robert Tibshirani,et al.  Model Search and Inference By Bootstrap "bumping , 1995 .

[9]  Kenji Fukumizu,et al.  Adaptive Method of Realizing Natural Gradient Learning for Multilayer Perceptrons , 2000, Neural Computation.

[10]  Xin Yao,et al.  Ensemble learning via negative correlation , 1999, Neural Networks.

[11]  Jason P. Mitchell,et al.  Feeling-of-knowing in episodic memory: an event-related fMRI study , 2003, NeuroImage.

[12]  L. Reder,et al.  What determines initial feeling of knowing? Familiarity with question terms, not with the answer , 1992 .

[13]  H. Jaap van den Herik,et al.  Interpretable Neural Networks with BP-SOM , 1998, ECML.

[14]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[15]  Philip J. Stroffolino,et al.  To calculate or not to calculate: A source activation confusion model of problem familiarity's role in strategy selection , 1997 .

[16]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[17]  Kenji Fukumizu,et al.  Adaptive natural gradient learning algorithms for various stochastic models , 2000, Neural Networks.

[18]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[19]  Shun-ichi Amari,et al.  Improving Generalization Performance of Natural Gradient Learning Using Optimized Regularization by NIC , 2004, Neural Computation.