Electrostatic field framework for supervised and semi-supervised learning from incomplete data

In this paper a classification framework for incomplete data, based on electrostatic field model is proposed. An original approach to exploiting incomplete training data with missing features, involving extensive use of electrostatic charge analogy, has been used. The framework supports a hybrid supervised and unsupervised training scenario, enabling learning simultaneously from both labelled and unlabelled data using the same set of rules and adaptation mechanisms. Classification of incomplete patterns has been facilitated by introducing a local dimensionality reduction technique, which aims at exploiting all available information using the data ‘as is’, rather than trying to estimate the missing values. The performance of all proposed methods has been extensively tested in a wide range of missing data scenarios, using a number of standard benchmark datasets in order to make the results comparable with those available in current and future literature. Several modifications to the original Electrostatic Field Classifier aiming at improving speed and robustness in higher dimensional spaces have also been introduced and discussed.

[1]  K.J.Resch,et al.  Experimental One-Way Quantum Computing , 2005, quant-ph/0503126.

[2]  Michael I. Jordan,et al.  Supervised learning from incomplete data via an EM approach , 1993, NIPS.

[3]  Bogdan Gabrys,et al.  Physical field models for pattern classification , 2003, Soft Comput..

[4]  S. Turner,et al.  The SAGE Handbook of Social Science Methodology , 2007 .

[5]  Andrew McCallum,et al.  Toward Optimal Active Learning through Sampling Estimation of Error Reduction , 2001, ICML.

[6]  D. Rubin.  Multiple imputation for nonresponse in surveys , 1989 .

[7]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[8]  John W. Fisher,et al.  Learning from Examples with Information Theoretic Criteria , 2000, J. VLSI Signal Process..

[9]  W. H. Zurek.  Complexity, Entropy and the Physics of Information , 1990 .

[10]  Michael C. Mozer,et al.  Coulomb Classifiers: Reinterpreting SVMs as Electrostatic Systems ; CU-CS-921-01 , 2001 .

[11]  Tom Michael Mitchell,et al.  The Role of Unlabeled Data in Supervised Learning , 2004 .

[12]  D. Rubin.  INFERENCE AND MISSING DATA , 1975 .

[13]  Yan Zhou,et al.  Enhancing Supervised Learning with Unlabeled Data , 2000, ICML.

[14]  Witold Pedrycz,et al.  Fuzzy clustering with partial supervision , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[15]  Bogdan Gabrys,et al.  Electrostatic Field Classifier for Deficient Data , 2009, Computer Recognition Systems 3.

[16]  Bogdan Gabrys,et al.  Neuro-fuzzy approach to processing inputs with missing values in pattern recognition problems , 2002, Int. J. Approx. Reason..

[17]  Charu C. Aggarwal,et al.  On the Surprising Behavior of Distance Metrics in High Dimensional Spaces , 2001, ICDT.

[18]  Ingram Olkin,et al.  Incomplete data in sample surveys , 1985 .

[19]  J. Schafer,et al.  Missing data: our view of the state of the art. , 2002, Psychological methods.

[20]  Ingram Olkin,et al.  Incomplete data in sample surveys. Vol. 3: proceedings of the symposium , 1983 .

[21]  Klaus Obermayer,et al.  Coulomb Classifiers: Generalizing Support Vector Machines via an Analogy to Electrostatic Systems , 2002, NIPS.

[22]  Charu C. Aggarwal,et al.  Re-designing distance functions and distance-based applications for high dimensional data , 2001, SGMD.

[23]  Deniz Erdoğmuş,et al.  Blind source separation using Renyi's mutual information , 2001, IEEE Signal Processing Letters.

[24]  Jonathan Goldstein,et al.  When Is ''Nearest Neighbor'' Meaningful? , 1999, ICDT.

[25]  Stefan C. Kremer,et al.  Clustering unlabeled data with SOMs improves classification of labeled real-world data , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[26]  Matthias Seeger,et al.  Learning from Labeled and Unlabeled Data , 2010, Encyclopedia of Machine Learning.

[27]  Mark S. Sherwin,et al.  Quantum Computation with Quantum Dots and Terahertz Cavity Quantum Electrodynamics , 1999 .

[28]  Deniz Erdogmus,et al.  Information Theoretic Learning , 2005, Encyclopedia of Artificial Intelligence.

[29]  Bogdan Gabrys,et al.  A framework for machine learning based on dynamic physical fields , 2009, Natural Computing.

[30]  Patricio Cumsille,et al.  4 Methods for Handling Missing Data , 2012 .

[31]  Michel Verleysen,et al.  Non-Euclidean metrics for similarity search in noisy datasets , 2005, ESANN.

[32]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[33]  Joseph L Schafer,et al.  Inference with Imputed Conditional Means , 2000 .

[34]  Volker Tresp,et al.  Training Neural Networks with Deficient Data , 1993, NIPS.

[35]  Kamal Nigam,et al.  Understanding the Behavior of Co-training , 2000, KDD 2000.

[36]  Robert P. W. Duin,et al.  A Matlab Toolbox for Pattern Recognition , 2004 .

[37]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[38]  Warren Sarle.  Prediction with Missing Inputs , 1998 .

[39]  Robert Jenssen,et al.  Some Equivalences between Kernel Methods and Information Theoretic Methods , 2006, J. VLSI Signal Process..

[40]  Bogdan Gabrys,et al.  Combining labelled and unlabelled data in the design of pattern classification systems , 2004, Int. J. Approx. Reason..

[41]  Dimitrios Gunopulos,et al.  Applying Electromagnetic Field Theory Concepts to Clustering with Constraints , 2009, ECML/PKDD.

[43]  Kari Torkkola,et al.  Feature Extraction by Non-Parametric Mutual Information Maximization , 2003, J. Mach. Learn. Res..

[44]  Ludmila I. Kuncheva,et al.  Fuzzy Classifier Design , 2000, Studies in Fuzziness and Soft Computing.

[45]  Adam Paszkiewicz,et al.  On quantum information , 2012, ArXiv.

[46]  Stephen M. Barnett,et al.  Quantum information , 2005, Acta Physica Polonica A.