PRISM – A novel framework for pattern recognition

In this paper, we introduce a new model of solving pattern recognition tasks called PRISM (Pattern Recognition using Information Slicing Method). The main concept behind PRISM is the slicing of information through multiple planes across different feature axes to generate a number of cells. The number of cells created and their volume depends upon the number partitions per axes. In this context we define resolution as the number of partitions per axes. In this paper, we make the following contributions. First, we provide a brief survey of the class separability measures and feature partitioning schemes used for pattern recognition. Secondly, we define the PRISM framework and the algorithm for data assignment to cells. Thirdly, we detail four important concepts in PRISM: purity, neighbourhood separability, collective entropy, and data compactness. The first two measures define the data complexity, the next measure relates to uncertainty, and the last measure defines the alternative to statistical data variance in the PRISM framework. Fourthly, we investigate the variability in the estimates of these measures depending on the placement of partitions on each feature axis. Finally, we give an overview of experimental successes achieved with PRISM in the areas of classification complexity estimation and feature selection.

[1]  Moshe Ben-Bassat,et al.  35 Use of distance measures, information measures and error bounds in feature evaluation , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[2]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[3]  Leszek Plaskota,et al.  Information complexity of neural networks , 2000, Neural Networks.

[4]  C. W. Therrien,et al.  Decision, Estimation and Classification: An Introduction to Pattern Recognition and Related Topics , 1989 .

[5]  Godfried T. Toussaint,et al.  Bibliography on estimation of misclassification , 1974, IEEE Trans. Inf. Theory.

[6]  T. Cacoullos.  Estimation of a multivariate density , 1966 .

[7]  C. S. Wallace,et al.  An Information Measure for Classification , 1968, Comput. J..

[8]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[9]  N. Glick.  Separation and probability of correct classification among two or more distributions , 1973 .

[10]  V. Leitáo,et al.  Computer Graphics: Principles and Practice , 1995 .

[11]  Lalit M. Patnaik,et al.  Genetic programming based pattern classification with feature space partitioning , 2001, Inf. Sci..

[12]  So Young Sohn,et al.  Meta Analysis of Classification Algorithms for Pattern Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  David G. Stork,et al.  Pattern Classification , 1973 .

[14]  Donald Meagher,et al.  Geometric modeling using octree encoding , 1982, Computer Graphics and Image Processing.

[15]  King-Sun Fu,et al.  Handbook of pattern recognition and image processing , 1986 .

[16]  Stanley C. Ahalt,et al.  Theoretical and complexity issues for feature set evaluation using boundary methods , 1998, Defense, Security, and Sensing.

[17]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[18]  W. Pierson,et al.  Using boundary methods for estimating class separability , 1998 .

[19]  Uri Lipowezky.  Selection of the optimal prototype subset for 1-NN classification , 1998, Pattern Recognit. Lett..

[20]  Christian Böhm,et al.  Searching in high-dimensional spaces: Index structures for improving the performance of multimedia databases , 2001, CSUR.

[21]  Venu Govindaraju,et al.  Improvements in K-Nearest Neighbor Classification , 2001, ICAPR.

[22]  E. Dodd,et al.  The Mathematical Theory of Probabilities. , 1923 .

[23]  Sarunas Raudys,et al.  On Dimensionality, Sample Size, Classification Error, and Complexity of Classification Algorithm in Pattern Recognition , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Chris L. Jackins,et al.  Oct-trees and their use in representing three-dimensional objects , 1980 .

[25]  Tin Kam Ho,et al.  Complexity of Classification Problems and Comparative Advantages of Combined Classifiers , 2000, Multiple Classifier Systems.

[26]  John Maddox.  Complicated measures of complexity , 1990, Nature.

[28]  Youshou Wu,et al.  Classification complexity and its estimation algorithm for two-class classification problem , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[29]  C. Quesenberry,et al.  A nonparametric estimate of a multivariate density function , 1965 .

[30]  A. Figueiras-Vidal,et al.  Boundary methods for distribution analysis , 1997 .

[31]  Werner Dilger,et al.  Feature Space Partitioning By Non-Linear And Fuzzy Decision Trees , 1997 .

[32]  Fuad Rahman,et al.  Measuring classification complexity of image databases: a novel approach , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[33]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[34]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[35]  Arnold C. Williams.  Comparative study of the Gaussian form of the Bhattacharyya metric for ATR performance evaluation , 1997, Defense, Security, and Sensing.

[36]  Rabab Kreidieh Ward,et al.  Vector Quantization Technique for Nonparametric Classifier Design , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[38]  Gregory Michael Hunter,et al.  Efficient computation and data structures for graphics. , 1978 .

[39]  Sameer Singh,et al.  Multiresolution Estimates of Classification Complexity , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  H. L. Rietz.  The Mathematical Theory of Probabilities , 1916 .

[41]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[42]  J. Crutchfield,et al.  Measures of statistical complexity: Why? , 1998 .

[43]  C. H. Chen,et al.  On information and distance measures, error bounds, and feature selection , 1976, Information Sciences.

[44]  SohnSo Young.  Meta Analysis of Classification Algorithms for Pattern Recognition , 1999 .

[45]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[46]  Miguel Oliveira e Silva,et al.  A class discriminality measure based on feature space partitioning , 1996, Pattern Recognit..