Training a feed-forward network with incomplete data due to missing input variables

Abstract Data available for training a neural network may be deficient not only in quantity of data but entire independent variables with their data may be missing such as is often the situation for software engineering data. This may cause the relation based on the available data to exhibit the property of one-to-many (o-m) valuedness or almost o-m valuedness. Multiplayer perceptrons or feed-forward network however are generally trained to represent functions or m-o mappings. The solution consists of adding another input to the standard feed-forward network and of modifying the training algorithm to allow for determination of this input for which no training data is available. If the values for the additional input are restricted to a discrete set then they may be perceived as cluster identifiers and the training method may be perceived as another form of clustering or segmentation of the input. If the missing input variable is assumed to have a finite number of values the method proposed here may be compared to mixture of experts and mixture density networks except that the proposed method is more direct solution. The modified feed-forward network and training method has been successfully applied to several examples.

[1]  Roelof K. Brouwer.  Training a feed-forward network by feeding gradients forward rather than by back-propagation of errors , 1997, Neurocomputing.

[2]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[3]  Naohiro Fukumura,et al.  A computational model for recognizing objects and planning hand shapes in grasping movements , 1995, Neural Networks.

[4]  Kenji Nakayama,et al.  Approximating many valued mappings using a recurrent neural network , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[5]  Kazuyuki Hiraoka,et al.  Recalling of Many-Valued Functions by Successive Iteration on Bottleneck Networks , 1998, ICONIP.

[6]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[7]  Tong Lee,et al.  Design of neural networks for multi-value regression , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[8]  Tony Plate.  Avoiding Roundoff Error in Backpropagating Derivatives , 1996, Neural Networks: Tricks of the Trade.

[9]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[10]  S. Srihari.  Mixture Density Networks , 1994 .

[11]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .