A diffusion-neural-network for learning from small samples

Abstract Neural information processing models largely assume that the patterns for training a neural network are sufficient. Otherwise, there must exist a non-negligible error between the real function and the estimated function from a trained network. To reduce the error, in this paper, we suggest a diffusion-neural-network (DNN) to learn from a small sample consisting of only a few patterns. A DNN with more nodes in the input and layers is trained by using the deriving patterns instead of original patterns. In this paper, we give an example to show how to construct a DNN for recognizing a non-linear function. In our case, the DNN’s error is less than the error of the conventional BP network, about 48%. To substantiate the special case arguments, we also study other two non-linear functions with simulation technology. The results show that the DNN model is very effective in the case where the target function has a strong non-linearity or a given sample is very small.

[1]  Andries P. Engelbrecht,et al.  Optimizing the number of hidden nodes of a feedforward artificial neural network , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[2]  Bagrat R. Amirikian,et al.  What size network is good for generalization of a specific task of interest? , 1994, Neural Networks.

[3]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[4]  Ky Van Ha.  Hierarchical radial basis function networks , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[5]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[6]  Yoh-Han Pao,et al.  Adaptive pattern recognition and neural networks , 1989 .

[7]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[8]  Patrick K. Simpson,et al.  Artificial Neural Systems: Foundations, Paradigms, Applications, and Implementations , 1990 .

[9]  Da Ruan,et al.  Information Diffusion Principle and Application in Fuzzy Neuron , 1996 .

[10]  Julian Morris,et al.  A procedure for determining the topology of multilayer feedforward neural networks , 1994, Neural Networks.

[11]  Yee Leung,et al.  Estimating the relationship between isoseismal area and earthquake magnitude by a hybrid fuzzy-neural-network method , 1999, Fuzzy Sets Syst..

[12]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[13]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[14]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[15]  Yih-Fang Huang,et al.  Bounds on the number of hidden neurons in multilayer perceptrons , 1991, IEEE Trans. Neural Networks.

[16]  Chongfu Huang,et al.  Information Diffusion Techniques and Small-Sample Problem , 2002, Int. J. Inf. Technol. Decis. Mak..

[17]  L. Zadeh.  Fuzzy sets as a basis for a theory of possibility , 1999 .

[18]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[19]  Songwu Lu,et al.  Robust nonlinear system identification using neural-network models , 1998, IEEE Trans. Neural Networks.

[20]  Gary G. R. Green,et al.  Neural networks, approximation theory, and finite precision computation , 1995, Neural Networks.

[21]  Huang Chongfu,et al.  Deriving samples from incomplete data , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[22]  H. Schioler,et al.  Estimating conditional distributions by neural networks , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[23]  Halbert White,et al.  Connectionist nonparametric regression: Multilayer feedforward networks can learn arbitrary mappings , 1990, Neural Networks.

[24]  Yong Shi,et al.  Towards Efficient Fuzzy Information Processing - Using the Principle of Information Diffusion , 2002, Studies in Fuzziness and Soft Computing.

[25]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[26]  Włodzisław Duch,et al.  Quo vadis, computational intelligence? , 2004 .

[27]  Huang Chong-fu,et al.  Principle of information diffusion , 1997 .

[28]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[29]  Chongfu Huang,et al.  Principle of information diffusion , 1997, Fuzzy Sets Syst..

[30]  Patrick van der Smagt.  Minimisation methods for training feedforward neural networks , 1994, Neural Networks.