Neural Network Learning from Ambiguous Training Data

After a brief review of the different types and causes of ambiguous training data and the problems of learning from such data, a class of multi-target models are presented which suggest that neural networks are even better at solving these problems than previously realized. They are able to learn which non-ambiguous subset of a larger ambiguous set of training data best captures any underlying regularities in that data and hence optimize generalization while minimizing the problems of overtraining. It is also shown how the deliberate generation of ambiguous training data can begin to solve some of the long-standing representational problems of mapping time sequences, such as the alignment problem for reading and spelling. The general ideas are illustrated throughout with the well- known problem of tex-to-phoneme conversion, and detailed results of a range of neural network simulations are presented.