On comparing the performance of object recognition systems

We give a methodology for the evaluation and comparison of object recognition systems. The methodology is based on indicators of two kinds: (1) statistical and (2) algorithmic. Statistical indicators measure the significance of the performance difference between different systems and rank the performances when the difference is significant. Of the various statistical indicators, we use the Kruskal-Wallis H test. Algorithmic indicators include the usual space and time complexity measures, and various performance curves in variables of error and test data sample size. We illustrate the methodology to evaluate a number of pattern classifiers.

[1]  Chuen-Tsai Sun,et al.  Neuro-fuzzy And Soft Computing: A Computational Approach To Learning And Machine Intelligence [Books in Brief] , 1997, IEEE Transactions on Neural Networks.

[2]  Karl-Friedrich Kraiss,et al.  Pixel based 3D object recognition with bidirectional associative memories , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[3]  B. Achiriloaie,et al.  VI REFERENCES , 1961 .

[4]  Rui J. P. de Figueiredo,et al.  A general moment-invariants/attributed-graph method for three-dimensional object recognition from a single image , 1986, IEEE J. Robotics Autom..

[5]  Stephen Grossberg,et al.  Art 2: Self-Organization Of Stable Category Recognition Codes For Analog Input Patterns , 1988, Other Conferences.

[6]  Wei-Chung Lin,et al.  A hierarchical multiple-view approach to three-dimensional object recognition , 1991, IEEE Trans. Neural Networks.

[7]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[8]  Demetri Psaltis,et al.  Image Normalization by Complex Moments , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Nasser M. Nasrabadi,et al.  Object recognition by a Hopfield neural network , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[10]  D. Wolfe,et al.  Nonparametric Statistical Methods. , 1974 .

[11]  Jyh-Shing Roger Jang,et al.  Fuzzy Modeling Using Generalized Neural Networks and Kalman Filter Algorithm , 1991, AAAI.

[12]  David Vernon,et al.  Machine vision - automated visual inspection and robot vision , 1991 .

[13]  Gail A. Carpenter,et al.  ART-EMAP: A neural network architecture for object recognition by evidence accumulation , 1995, IEEE Trans. Neural Networks.

[14]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[15]  S. B. Kang,et al.  3-D object recognition using fuzzy quaternions , 1992 .

[16]  Ramesh C. Jain,et al.  Three-dimensional object recognition , 1985, CSUR.

[17]  H H Bülthoff,et al.  How are three-dimensional objects represented in the brain? , 1994, Cerebral cortex.

[18]  Gian Luca Foresti,et al.  3D object recognition by neural trees , 1997, Proceedings of International Conference on Image Processing.

[19]  Detlef Nauck,et al.  Foundations Of Neuro-Fuzzy Systems , 1997 .

[20]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[21]  Alireza Khotanzad,et al.  Invariant Image Recognition by Zernike Moments , 1990, IEEE Trans. Pattern Anal. Mach. Intell..