Design of neural networks for multi-value regression

The problem of multi-value regression estimation with neural network architecture is addressed. We only consider feedforward neural networks (FNN) and we also confine the multi-value regression problems to those mapping N input variables to a single output, while the numbers of output values may be different for different input. We propose a modular neural network approach to solve this problem with each module handling a sub-range of the original one such that each module now only handles a many-to-one or one-to-one regression estimation. With such an approach, a verification process is necessary to determine which module provides the correct output value and two implementations are discussed. Several examples are used to illustrate the proposed method.

[1]  Ian T. Nabney,et al.  Regularisation of mixture density networks , 1999 .

[2]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[3]  M. Shizawa,et al.  Regularization networks for approximating multi-valued functions: learning ambiguous input-output mappings from examples , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[4]  Ian T. Nabney,et al.  Bayesian training of mixture density networks , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[5]  Ah Chung Tsoi,et al.  Universal Approximation Using Feedforward Neural Networks: A Survey of Some Existing Methods, and Some New Results , 1998, Neural Networks.