Generalisations of stochastic supervision models

Abstract When the labelling information is not deterministic, traditional supervised learning algorithms cannot be applied. In this case, stochastic supervision models provide a valuable alternative to classification. However, these models are restricted in several aspects, which critically limits their applicability. In this paper, we provide four generalisations of stochastic supervision models, extending them to asymmetric assessments, multiple classes, feature-dependent assessments and multi-modal classes, respectively. Corresponding to these generalisations, we derive four new EM algorithms. We show the effectiveness of our generalisations through illustrative examples of simulated datasets, as well as real-world examples of three famous datasets, the MNIST dataset, the CIFAR-10 dataset and the EMNIST dataset.

[1]  Charles Bouveyron,et al.  Robust supervised classification with mixture models: Learning from data with uncertain labels , 2009, Pattern Recognit..

[2]  H. Theil,et al.  Economic Forecasts and Policy. , 1959 .

[3]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[4]  M. Verleysen,et al.  Classification in the Presence of Label Noise: A Survey , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[5]  D. Titterington.  Some recent research in the analysis of mixture distributions , 1990 .

[6]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[7]  Gregory Cohen,et al.  EMNIST: Extending MNIST to handwritten letters , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[8]  Terence J. O'Neill.  Normal Discrimination with Unclassified Observations , 1978 .

[9]  Friedhelm Schwenker,et al.  Pattern classification and clustering: A review of partially supervised learning approaches , 2014, Pattern Recognit. Lett..

[10]  Adrian E. Raftery,et al.  Model-Based Clustering, Discriminant Analysis, and Density Estimation , 2002 .

[11]  P. Deb.  Finite Mixture Models , 2008 .

[12]  John Aitchison,et al.  The Statistical Analysis of Compositional Data , 1986 .

[13]  Eric Granger,et al.  Multiple instance learning: A survey of problem characteristics and applications , 2016, Pattern Recognit..

[14]  R. Tibshirani,et al.  Discriminant Analysis by Gaussian Mixtures , 1996 .

[15]  Friedhelm Schwenker,et al.  Partially supervised learning for pattern recognition , 2014, Pattern Recognit. Lett..

[16]  John Aitchison,et al.  Statistical diagnosis when basic cases are not classified with certainty , 1976 .

[17]  Ryan P. Browne,et al.  Model-Based Learning Using a Mixture of Mixtures of Gaussian and Uniform Distributions , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Subhas C. Nandy,et al.  Efficiency of logistic-normal stochastic supervision , 1990, Pattern Recognit..

[19]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[20]  C. B. Chittineni.  Learning with imperfectly labeled patterns , 1980, Pattern Recognit..

[21]  Subhas C. Nandy,et al.  Efficiency of discriminant analysis when initial samples are classified stochastically , 1990, Pattern Recognit..

[22]  Subhas C. Nandy,et al.  Discriminant analysis with a stochastic supervisor , 1987, Pattern Recognit..

[23]  Trevor Hastie,et al.  An Introduction to Statistical Learning , 2013, Springer Texts in Statistics.

[24]  T. Krishnan.  Efficiency of learning with imperfect supervision , 1988, Pattern Recognit..

[25]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[26]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[27]  G. McLachlan.  Iterative Reclassification Procedure for Constructing An Asymptotically Optimal Rule of Allocation in Discriminant-Analysis , 1975 .

[28]  D. M. Titterington.  An alternative stochastic supervisor in discriminant analysis , 1989, Pattern Recognit..

[29]  T. Krishnan,et al.  Pattern recognition with an imperfect supervisor , 1989, Pattern Recognit..

[30]  Geoffrey J. McLachlan,et al.  Deep Gaussian mixture models , 2017, Statistics and Computing.