The dissimilarity space: Bridging structural and statistical pattern recognition

Highlights? Consciousness divides human recognition in structural and statistical approaches. ? Dissimilarities, fundamental in human recognition are suited to integrate the two. ? The dissimilarity space is a good vector space for the dissimilarity representation. ? Classifiers in dissimilarity space are accurate and/or complexity. ? Combining dissimilarities by averaging may improve results further. Human experts constitute pattern classes of natural objects based on their observed appearance. Automatic systems for pattern recognition may be designed on a structural description derived from sensor observations. Alternatively, training sets of examples can be used in statistical learning procedures. They are most powerful for vectorial object representations. Unfortunately, structural descriptions do not match well with vectorial representations. Consequently it is difficult to combine the structural and statistical approaches to pattern recognition.Structural descriptions may be used to compare objects. This leads to a set of pairwise dissimilarities from which vectors can be derived for the purpose of statistical learning. The resulting dissimilarity representation bridges thereby the structural and statistical approaches.The dissimilarity space is one of the possible spaces resulting from this representation. It is very general and easy to implement. This paper gives a historical review and discusses the properties of the dissimilarity space approaches illustrated by a set of examples on real world datasets.

[1]  Daphna Weinshall,et al.  Classification with Nonmetric Distances: Image Retrieval and Class Representation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. I. , 1962 .

[3]  N. JARDINE,et al.  A New Approach to Pattern Recognition , 1971, Nature.

[4]  Alexander J. Smola,et al.  Learning with non-positive kernels , 2004, ICML.

[5]  Hans Burkhardt,et al.  Invariant kernel functions for pattern analysis and machine learning , 2007, Machine Learning.

[6]  Robert P. W. Duin,et al.  Feature-Based Dissimilarity Space Classification , 2010, ICPR Contests.

[7]  Casimir A. Kulikowski,et al.  Featureless pattern recognition in an imaginary Hilbert space , 2002, Object recognition supported by user interaction for service robots.

[8]  Shimon Edelman,et al.  Representation and recognition in vision , 1999 .

[9]  Kaspar Riesen,et al.  Graph Embedding in Vector Spaces by Means of Prototype Selection , 2007, GbRPR.

[10]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[11]  Satosi Watanabe,et al.  Pattern Recognition: Human and Mechanical , 1985 .

[12]  Oleg Golubitsky,et al.  What is a structural representation ? Fourth variation ∗ , 2005 .

[13]  Edwin R. Hancock,et al.  Spherical Embedding and Classification , 2010, SSPR/SPR.

[14]  Klaus Obermayer,et al.  Support Vector Machines for Dyadic Data , 2006, Neural Computation.

[15]  Bernard Haasdonk,et al.  Feature space interpretation of SVMs with indefinite kernels , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[17]  Azriel Rosenfeld,et al.  Progress in pattern recognition , 1985 .

[18]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[19]  Robert P. W. Duin,et al.  Prototype selection for dissimilarity-based classifiers , 2006, Pattern Recognit..

[20]  Filiberto Pla,et al.  Experimental study on prototype optimisation algorithms for prototype-based classification in vector spaces , 2006, Pattern Recognit..

[21]  Robert P. W. Duin,et al.  Dissimilarity-based classification for vectorial representations , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[22]  Robert P. W. Duin,et al.  On Combining Dissimilarity Representations , 2001, Multiple Classifier Systems.

[23]  Casimir A. Kulikowski,et al.  Featureless Pattern Recognition in an Imaginary Hilbert Space and Its Application to Protein Fold Classification , 2001, MLDM.

[24]  King-Sun Fu,et al.  Syntactic Pattern Recognition And Applications , 1968 .

[25]  Ethem Alpaydin,et al.  Combining multiple representations and classifiers for pen-based handwritten digit recognition , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[26]  Robert P. W. Duin,et al.  Beyond Traditional Kernels: Classification in Two Dissimilarity-Based Representation Spaces , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[27]  R. Duin,et al.  The dissimilarity representation for pattern recognition , a tutorial , 2009 .

[28]  S. Canu,et al.  Functional learning through kernel , 2002 .

[29]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. II , 1962 .

[30]  Robert P. W. Duin,et al.  The Dissimilarity Representation for Pattern Recognition - Foundations and Applications , 2005, Series in Machine Perception and Artificial Intelligence.

[31]  Elzbieta Pekalska,et al.  Kernel Discriminant Analysis for Positive Definite and Indefinite Kernels , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Robert P. W. Duin,et al.  Prototype Selection for Dissimilarity Representation by a Genetic Algorithm , 2010, 2010 20th International Conference on Pattern Recognition.

[33]  J. Kruskal.  Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[34]  S. Canu,et al.  M L ] 6 O ct 2 00 9 Functional learning through kernel , 2009 .

[35]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[36]  Horst Bunke,et al.  Transforming Strings to Vector Spaces Using Prototype Selection , 2006, SSPR/SPR.