Pattern recognition with ordered labels

Abstract We consider a multi-class pattern recognition problem with linearly ordered labels and a loss function, which is a sum of squared deviations of decisions from true classes. It is proved that the optimal, in the Bayesian sense, decision rule is the a posteriori mean, which is then rounded to the nearest integer. Then, the plug-in type estimator of the optimal decision rule is proposed. We also investigate its asymptotic properties. Finally, we discuss briefly the structure of a neural net, which approximates the optimal decision rule.

[1]  Adam Krzyzak,et al.  Fast k-NN classification rule using metric on space-filling curves , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[2]  L. Devroye,et al.  Nonparametric density estimation : the L[1] view , 1987 .

[3]  Ewa Skubalska-Rafajlowicz,et al.  Pattern recognition algorithms based on space-filling curves and orthogonal expansions , 2001, IEEE Trans. Inf. Theory.

[4]  Gábor Lugosi,et al.  Nonparametric estimation via empirical risk minimization , 1995, IEEE Trans. Inf. Theory.

[5]  Nicolaos B. Karayiannis,et al.  On the construction and training of reformulated radial basis function neural networks , 2003, IEEE Trans. Neural Networks.

[6]  Ewa Skubalska-Rafajlowicz,et al.  RBF Neural Network for Probability Density Function Estimation and Detecting Changes in Multivariate Processes , 2006, ICAISC.

[7]  Ewaryst Rafajlowicz,et al.  Improving the Efficiency of Counting Defects by Learning RBF Nets with MAD Loss , 2006, ICAISC.

[8]  M. Pawlak,et al.  Pattern Classification with Noisy Features , 1998, SSPR/SPR.

[9]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[10]  Adam Krzyzak,et al.  On radial basis function nets and kernel regression: Statistical consistency, convergence rates, and receptive field size , 1994, Neural Networks.

[11]  Michel Loève,et al.  Probability Theory I , 1977 .

[12]  Ansgar Steland,et al.  Nonlinear Image Processing and Filtering: A Unified Approach Based on Vertically Weighted Regression , 2008, Int. J. Appl. Math. Comput. Sci..

[13]  Ewaryst Rafajlowicz,et al.  RBF Nets in Faults Localization , 2006, ICAISC.

[14]  Ioannis Pitas,et al.  Median radial basis function neural network , 1996, IEEE Trans. Neural Networks.

[15]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[16]  W. Cholewa,et al.  Fault Diagnosis: Models, Artificial Intelligence, Applications , 2004 .

[17]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[18]  E. Skubalska-Rafajłowicz.  Data compression for pattern recognition based on space-filling curve pseudo-inverse mapping , 2001 .

[19]  Miroslaw Pawlak,et al.  Necessary and sufficient conditions for Bayes risk consistency of a recursive kernel classification rule , 1987, IEEE Trans. Inf. Theory.

[20]  Luc Devroye,et al.  Nonparametric Density Estimation , 1985 .

[21]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[22]  Robert M. Pap,et al.  Fault Diagnosis , 1990, Bayesian Networks in Fault Diagnosis.

[23]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[24]  Ewa Skubalska-Rafajlowicz.  Local Correlation and Entropy Maps as Tools for Detecting Defects in Industrial Images , 2008, Int. J. Appl. Math. Comput. Sci..

[25]  中澤 真,et al.  Devroye, L., Gyorfi, L. and Lugosi, G. : A Probabilistic Theory of Pattern Recognition, Springer (1996). , 1997 .

[26]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[27]  Adam Krzyzak,et al.  Combining Space-Filling Curves and Radial Basis Function Networks , 2004, ICAISC.