Using unlabeled data for learning classification problems

This chapter presents an approach of using unlabeled data for learning classification problems. The chapter consists of two parts. In the first part of the chapter, an approach of using both labeled and unlabeled data to train a multilayer percetron is presented. The approach banks on the assumption that regions of low pattern density usually separate data classes. The unlabeled data are iteratively preprocessed by a perceptron being trained to obtain the soft class label estimates. It is demonstrated that substantial gains in classification performance may be achieved by using the approach when the labeled data do not adequately represent the entire class distributions. In the second part of the chapter, we propose a quality function for learning decision boundary between data clusters from unlabeled data. The function is based on third order polynomials. The objective of the quality function is to find a place in the input sparse in data points. By maximizing the quality function, we find a decision boundary between data clusters. A superiority of the proposed quality function over the other similar functions as well as the conventional clustering algorithms tested has been observed in the experiments.

[1]  Erkki Oja,et al.  One-unit Learning Rules for Independent Component Analysis , 1996, NIPS.

[2]  Antanas Verikas,et al.  A Random Search Technique for Training Neural Networks , 1997, ICONIP.

[3]  Nathan Intrator,et al.  Objective function formulation of the BCM theory of visual cortical plasticity: Statistical connections, stability conditions , 1992, Neural Networks.

[4]  Clement T. Yu,et al.  Segmentation of skin cancer images , 1999, Image Vis. Comput..

[5]  James C. Bezdek,et al.  Fuzzy Kohonen clustering networks , 1994, Pattern Recognit..

[6]  Eric A. Wan,et al.  Neural network classification: a Bayesian interpretation , 1990, IEEE Trans. Neural Networks.

[7]  Sankar K. Pal,et al.  Neural network, self-organization and object extraction , 1992, Pattern Recognit. Lett..

[8]  Athanasios Kehagias,et al.  Time-Series Segmentation Using Predictive Modular Neural Networks , 1997, Neural Computation.

[9]  David J. Miller,et al.  Combined Learning and Use for a Mixture Model Equivalent to the RBF Classifier , 1998, Neural Computation.

[10]  David J. Miller,et al.  A Mixture of Experts Classifier with Learning Based on Both Labelled and Unlabelled Data , 1996, NIPS.

[11]  David A. Landgrebe,et al.  The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon , 1994, IEEE Trans. Geosci. Remote. Sens..

[12]  James C. Bezdek,et al.  Partially supervised clustering for image segmentation , 1996, Pattern Recognit..

[13]  Azriel Rosenfeld,et al.  Image enhancement and thresholding by optimization of fuzzy compactness , 1988, Pattern Recognit. Lett..

[14]  S. Grossberg,et al.  Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors , 1976, Biological Cybernetics.

[15]  P. Schmid.  Segmentation of digitized dermatoscopic images by two-dimensional color clustering , 1999, IEEE Transactions on Medical Imaging.

[16]  Joydeep Ghosh,et al.  Efficient Higher-Order Neural Networks for Classification and Function Approximation , 1992, Int. J. Neural Syst..

[17]  David A. Landgrebe,et al.  Asymptotic improvement of supervised learning by utilizing additional unlabeled samples: normal mixture density case , 1992, Optics & Photonics.

[18]  Mohan S. Kankanhalli,et al.  Color and spatial feature for content-based image retrieval , 1999, Pattern Recognit. Lett..

[19]  GrossbergS..  Adaptive pattern classification and universal recoding , 1976 .

[20]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[21]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[22]  David Levy,et al.  Book review: Neural Networks and Fuzzy Systems: A Dynamical Systems Approach to Machine Intelligence by Bart Kosko (Prentice Hall 1992) , 1992, CARN.

[23]  Bart Kosko,et al.  Neural networks and fuzzy systems: a dynamical systems approach to machine intelligence , 1991 .

[24]  Malik Magdon-Ismail,et al.  Incorporating Test Inputs into Learning , 1997, NIPS.

[25]  Mauro Barni,et al.  Colour-based detection of defects on chicken meat , 1997, Image Vis. Comput..

[26]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[27]  Mao-Jiun J. Wang,et al.  Image thresholding by minimizing the measures of fuzzines , 1995, Pattern Recognit..

[28]  Bruce W. Suter,et al.  The multilayer perceptron as an approximation to a Bayes optimal discriminant function , 1990, IEEE Trans. Neural Networks.

[29]  Michael G. Sowa,et al.  Tissue viability by multispectral near infrared imaging: a fuzzy C-means clustering analysis , 1998, IEEE Transactions on Medical Imaging.

[30]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[31]  Antanas Verikas,et al.  Colour image segmentation by modular neural network , 1997, Pattern Recognit. Lett..

[32]  Bart Kosko,et al.  Neural networks and fuzzy systems , 1998 .

[33]  Geoffrey G. Towell,et al.  Using Unlabeled Data for Supervised Learning , 1995, NIPS.

[34]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[35]  R.P. Lippmann,et al.  Pattern classification using neural networks , 1989, IEEE Communications Magazine.

[36]  Kazuhiro Nakano,et al.  Application of neural networks to the color grading of apples , 1997 .

[37]  Vittorio Castelli,et al.  On the exponential value of labeled samples , 1995, Pattern Recognit. Lett..

[38]  Antanas Verikas,et al.  A NEW METHOD FOR COLOUR MEASUREMENTS IN GRAPHIC ARTS , 1999 .

[39]  Amir F. Atiya.  An unsupervised learning technique for artificial neural networks , 1990, Neural Networks.

[40]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .