Classifier design with Parzen Windows

A number of methods are available in the literature to estimate the class-conditional densities for pattern classification. The Parzen-window method of density estimation is studied with emphasis on techniques for optimal window-width estimation. We report the window-widths obtained by using the BOOTSTRAP technique and compare them with MSE, MEISER and the LEAVE-ONE-OUT techniques. The performance of classifiers based on the Parzen window density estimate is compared with other well-known classifiers such as linear, quadratic, K-Nearest Neighbor, and binary tree on several real data sets.

[1]  Matthew Goldstein,et al.  Comparison of Some Density Estimate Classification Procedures , 1975 .

[2]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[3]  Luc Devroye,et al.  Automatic selection of a discrimination rule based upon minimization of the empirical risk , 1987 .

[4]  B. Efron,et al.  A Leisurely Look at the Bootstrap, the Jackknife, and , 1983 .

[5]  Robert P. W. Duin,et al.  On the Choice of Smoothing Parameters for Parzen Estimators of Probability Density Functions , 1976, IEEE Transactions on Computers.

[6]  Keinosuke Fukunaga,et al.  An Optimal Global Nearest Neighbor Metric , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Anil K. Jain,et al.  Bootstrap Techniques for Error Estimation , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  I. K. Sethi,et al.  Hierarchical Classifier Design Using Mutual Information , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Teuvo Kohonen,et al.  The 'neural' phonetic typewriter , 1988, Computer.

[10]  P. Diaconis,et al.  Computer-Intensive Methods in Statistics , 1983 .

[11]  Keinosuke Fukunaga,et al.  Bayes Error Estimation Using Parzen and k-NN Procedures , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Anil K. Jain,et al.  EXPERIMENT ON USING THE FRIEDMAN-RAFSKY TEST TO DETERMINE THE MULTIVARIATE NORMALITY OF A DATA SET. , 1985 .

[13]  John Van Ness,et al.  On the dominance of non-parametric Bayes rule discriminant algorithms in high dimensions , 1980, Pattern Recognit..

[14]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[15]  Larry D. Hostetler,et al.  Optimization of k nearest neighbor density estimates , 1973, IEEE Trans. Inf. Theory.

[16]  Anil K. Jain,et al.  39 Dimensionality and sample size considerations in pattern recognition practice , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[17]  Xiaobo Li,et al.  Tree classifier design with a permutation statistic , 1986, Pattern Recognit..

[18]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[19]  B. Efron.  Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation , 1983 .

[20]  Anil K. Jain,et al.  ON BALANCING DECISION FUNCTIONS. , 1979 .

[21]  B. Efron.  Bootstrap Methods: Another Look at the Jackknife , 1979 .