The Dissimilarity Representation for Structural Pattern Recognition

The patterns in collections of real world objects are often not based on a limited set of isolated properties such as features. Instead, the totality of their appearance constitutes the basis of the human recognition of patterns. Structural pattern recognition aims to find explicit procedures that mimic the learning and classification made by human experts in well-defined and restricted areas of application. This is often done by defining dissimilarity measures between objects and measuring them between training examples and new objects to be recognized. The dissimilarity representation offers the possibility to apply the tools developed in machine learning and statistical pattern recognition to learn from structural object representations such as graphs and strings. These procedures are also applicable to the recognition of histograms, spectra, images and time sequences taking into account the connectivity of samples (bins, wavelengths, pixels or time samples). The topic of dissimilarity representation is related to the field of non-Mercer kernels in machine learning but it covers a wider set of classifiers and applications. Recently much progress has been made in this area and many interesting applications have been studied in medical diagnosis, seismic and hyperspectral imaging, chemometrics and computer vision. This review paper offers an introduction to this field and presents a number of real world applications.

[1]  King-Sun Fu,et al.  Syntactic Pattern Recognition And Applications , 1968 .

[2]  Melanie Hilario,et al.  Learning to combine distances for complex representations , 2007, ICML '07.

[3]  Shimon Edelman,et al.  Representation and recognition in vision , 1999 .

[4]  Steven Gold,et al.  A Graduated Assignment Algorithm for Graph Matching , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  J. Bognár,et al.  Indefinite Inner Product Spaces , 1974 .

[6]  Robert P. W. Duin,et al.  Dissimilarity-Based Detection of Schizophrenia , 2010, ICPR 2010.

[7]  A Gordon,et al.  Classification, 2nd Edition , 1999 .

[8]  Wan-Jui Lee,et al.  An Inexact Graph Comparison Approach in Joint Eigenspace , 2008, SSPR/SPR.

[9]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[10]  N. JARDINE,et al.  A New Approach to Pattern Recognition , 1971, Nature.

[11]  Hans Burkhardt,et al.  Invariant kernel functions for pattern analysis and machine learning , 2007, Machine Learning.

[12]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[13]  Robert P. W. Duin,et al.  Feature-Based Dissimilarity Space Classification , 2010, ICPR Contests.

[14]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[15]  Horst Bunke,et al.  Transforming Strings to Vector Spaces Using Prototype Selection , 2006, SSPR/SPR.

[16]  Elzbieta Pekalska,et al.  Kernel Discriminant Analysis for Positive Definite and Indefinite Kernels , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Edwin R. Hancock,et al.  Structural, Syntactic, and Statistical Pattern Recognition, Joint IAPR International Workshop, SSPR&SPR 2010, Cesme, Izmir, Turkey, August 18-20, 2010. Proceedings , 2010, SSPR/SPR.

[18]  Robert P. W. Duin,et al.  Non-Euclidean Dissimilarities: Causes and Informativeness , 2010, SSPR/SPR.

[19]  Fabio Roli,et al.  Image Analysis and Processing - ICIAP 2005, 13th International Conference, Cagliari, Italy, September 6-8, 2005, Proceedings , 2005, ICIAP.

[20]  Robert P. W. Duin,et al.  Non-Euclidean Problems in Pattern Recognition Related to Human Expert Knowledge , 2010, ICEIS.

[21]  Robert P. W. Duin,et al.  The dissimilarity space: Bridging structural and statistical pattern recognition , 2012, Pattern Recognit. Lett..

[22]  J. A. Anderson,et al.  7 Logistic discrimination , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[23]  Wan-Jui Lee,et al.  A Study on Combining Sets of Differently Measured Dissimilarities , 2010, 2010 20th International Conference on Pattern Recognition.

[24]  Horst Bunke,et al.  Applications of approximate string matching to 2D shape recognition , 1993, Pattern Recognit..

[25]  P. Groenen,et al.  Modern multidimensional scaling , 1996 .

[26]  A. G. Arkad'ev,et al.  Computers and pattern recognition , 1967 .

[27]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. I. , 1962 .

[28]  Horst Bunke,et al.  Bridging the Gap between Graph Edit Distance and Kernel Machines , 2007, Series in Machine Perception and Artificial Intelligence.

[29]  Yi Liu,et al.  An Efficient Algorithm for Local Distance Metric Learning , 2006, AAAI.

[30]  Marcel J. T. Reinders,et al.  Sign Language Recognition by Combining Statistical DTW and Independent Classification , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  David G. Stork,et al.  Pattern Classification , 1973 .

[32]  Edwin R. Hancock,et al.  Geometric Characterisation of Graphs , 2005, ICIAP.

[33]  R. Duin,et al.  The dissimilarity representation for pattern recognition , a tutorial , 2009 .

[34]  Antonio Bellacicco,et al.  Handbook of statistics 2: Classification, pattern recognition and reduction of dimensionality: P.R. KRISHNAIAH and L.N. KANAL (Eds.) North-Holland, Amsterdam, 1982, xxii + 903 pages, Dfl.275.00 , 1984 .

[35]  Edwin R. Hancock,et al.  Spectral embedding of graphs , 2003, Pattern Recognit..

[36]  Robert P. W. Duin,et al.  Prototype Selection for Dissimilarity Representation by a Genetic Algorithm , 2010, 2010 20th International Conference on Pattern Recognition.

[37]  Robert P. W. Duin,et al.  The Dissimilarity Representation for Pattern Recognition - Foundations and Applications , 2005, Series in Machine Perception and Artificial Intelligence.

[38]  Joachim M. Buhmann,et al.  On the information and representation of non-Euclidean pairwise data , 2006, Pattern Recognit..

[39]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[40]  Arnold W. M. Smeulders,et al.  The Distribution Family of Similarity Distances , 2007, NIPS.

[41]  Casimir A. Kulikowski,et al.  Featureless Pattern Recognition in an Imaginary Hilbert Space and Its Application to Protein Fold Classification , 2001, MLDM.

[42]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[43]  Robert P. W. Duin,et al.  Clustering-Based Construction of Hidden Markov Models for Generative Kernels , 2009, EMMCVPR.

[44]  Robert P. W. Duin,et al.  Beyond Traditional Kernels: Classification in Two Dissimilarity-Based Representation Spaces , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[45]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[46]  Claus Bahlmann,et al.  The writer independent online handwriting recognition system frog on hand and cluster generative statistical dynamic time warping , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[48]  Bernard Haasdonk,et al.  Feature space interpretation of SVMs with indefinite kernels , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Sergios Theodoridis,et al.  Pattern Recognition, Fourth Edition , 2008 .

[50]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[51]  Robert P. W. Duin,et al.  Classification of three-way data by the dissimilarity representation , 2011, Signal Process..

[52]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. II , 1962 .

[53]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[54]  David H. Wolpert,et al.  Mathematics of Generalization: Proceedings: SFI-CNLS Workshop on Formal Approaches to Supervised Learning (1992: Santa Fe, N. M.) , 1995 .

[55]  Robert P. W. Duin,et al.  Component-based discriminative classification for hidden Markov models , 2009, Pattern Recognit..

[56]  Robert P. W. Duin,et al.  A generalization of dissimilarity representations using feature lines and feature planes , 2009, Pattern Recognit. Lett..

[57]  Elzbieta Pekalska,et al.  Indefinite Kernel Fisher Discriminant , 2008, 2008 19th International Conference on Pattern Recognition.

[58]  Robert P. W. Duin,et al.  Prototype selection for dissimilarity-based classifiers , 2006, Pattern Recognit..

[59]  Luis Alvarez,et al.  Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications , 2012, Lecture Notes in Computer Science.

[60]  Robert P. W. Duin,et al.  Dissimilarity representations allow for building good classifiers , 2002, Pattern Recognit. Lett..

[61]  Wan-Jui Lee,et al.  On Euclidean Corrections for Non-Euclidean Dissimilarities , 2008, SSPR/SPR.

[62]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[63]  Selim Aksoy,et al.  Recognizing Patterns in Signals, Speech, Images and Videos , 2010, Lecture Notes in Computer Science.

[64]  Satosi Watanabe,et al.  Pattern Recognition: Human and Mechanical , 1985 .

[65]  Robert P. W. Duin,et al.  On Improving Dissimilarity-Based Classifications Using a Statistical Similarity Measure , 2010, CIARP.

[66]  David H. Wolpert,et al.  The Mathematics of Generalization: The Proceedings of the SFI/CNLS Workshop on Formal Approaches to Supervised Learning , 1994 .

[67]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[68]  J. Kruskal.  Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[69]  Casimir A. Kulikowski,et al.  Featureless pattern recognition in an imaginary Hilbert space , 2002, Object recognition supported by user interaction for service robots.

[70]  Gabriela Andreu,et al.  Selecting the toroidal self-organizing feature maps (TSOFM) best organized to object recognition , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[71]  Kaspar Riesen,et al.  Graph Classification on Dissimilarity Space Embedding , 2008, SSPR/SPR.

[72]  Filiberto Pla,et al.  Experimental study on prototype optimisation algorithms for prototype-based classification in vector spaces , 2006, Pattern Recognit..

[73]  Edwin R. Hancock,et al.  Pattern Vectors from Algebraic Graph Theory , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[74]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[75]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[76]  Edwin R. Hancock,et al.  Spherical Embedding and Classification , 2010, SSPR/SPR.

[77]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[78]  R. Duin,et al.  Dissimilarity representation on functional spectral data for classification , 2011 .

[79]  Robert P. W. Duin,et al.  On refining dissimilarity matrices for an improved NN learning , 2008, 2008 19th International Conference on Pattern Recognition.

[80]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .