Efficient temporal pattern recognition by means of dissimilarity space embedding with discriminative prototypes

Dissimilarity space embedding (DSE) presents a method of representing data as vectors of dissimilarities. This representation is interesting for its ability to use a dissimilarity measure to embed various patterns (e.g. graph patterns with different topology and temporal patterns with different lengths) into a vector space. The method proposed in this paper uses a dynamic time warping (DTW) based DSE for the purpose of the classification of massive sets of temporal patterns. However, using large data sets introduces the problem of requiring a high computational cost. To address this, we consider a prototype selection approach. A vector space created by DSE offers us the ability to treat its independent dimensions as features allowing for the use of feature selection. The proposed method exploits this and reduces the number of prototypes required for accurate classification. To validate the proposed method we use two-class classification on a data set of handwritten on-line numerical digits. We show that by using DSE with ensemble classification, high accuracy classification is possible with very few prototypes. Graphical abstractDisplay Omitted HighlightsWe propose a method of using dissimilarity space embedding for temporal patterns.Ensemble classification was used for prototype selection to increase the efficiency.We evaluate the performance of our method on online handwritten digits.The experiments showed a high accuracy with a very small number of patterns.

[1]  Kaspar Riesen,et al.  Graph Embedding in Vector Spaces by Means of Prototype Selection , 2007, GbRPR.

[2]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[3]  Robert P. W. Duin,et al.  The Dissimilarity Representation for Pattern Recognition - Foundations and Applications , 2005, Series in Machine Perception and Artificial Intelligence.

[4]  Richard Bellman,et al.  Adaptive Control Processes - A Guided Tour (Reprint from 1961) , 2015, Princeton Legacy Library.

[5]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[6]  Antonio Torralba,et al.  Nonparametric Scene Parsing via Label Transfer , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Hiroshi Motoda,et al.  Feature Extraction, Construction and Selection , 1998 .

[8]  Robert Sabourin,et al.  On the Dissimilarity Representation and Prototype Selection for Signature-Based Bio-cryptographic Systems , 2013, SIMBAD.

[9]  Vittorio Murino,et al.  Encoding Structural Similarity by Cross-covariance Tensors for Image Classification , 2014, Int. J. Pattern Recognit. Artif. Intell..

[10]  Wei Fan,et al.  Bagging , 2009, Encyclopedia of Machine Learning.

[11]  Kaspar Riesen,et al.  Graph Classification Based on Vector Space Embedding , 2009, Int. J. Pattern Recognit. Artif. Intell..

[12]  Horst Bunke,et al.  Transforming Strings to Vector Spaces Using Prototype Selection , 2006, SSPR/SPR.

[13]  Chiranjib Bhattacharyya,et al.  Kernels for Large Margin Time-Series Classification , 2007, 2007 International Joint Conference on Neural Networks.

[14]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[15]  Robert P. W. Duin,et al.  The Dissimilarity Representation for Structural Pattern Recognition , 2011, CIARP.

[16]  Robert Sabourin,et al.  Applying Dissimilarity Representation to Off-Line Signature Verification , 2010, 2010 20th International Conference on Pattern Recognition.

[17]  Robert P. W. Duin,et al.  Metric Learning in Dissimilarity Space for Improved Nearest Neighbor Performance , 2014, S+SSPR.

[18]  Parham Aarabi,et al.  Tiny Videos: A Large Data Set for Nonparametric Video Retrieval and Frame Classification , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  J. Ross Quinlan,et al.  Bagging, Boosting, and C4.5 , 1996, AAAI/IAAI, Vol. 1.

[20]  Robert P. W. Duin,et al.  Dissimilarity representations allow for building good classifiers , 2002, Pattern Recognit. Lett..

[21]  Hiroshi Motoda,et al.  Feature Extraction, Construction and Selection: A Data Mining Perspective , 1998 .

[22]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[23]  Claus Bahlmann,et al.  Online handwriting recognition with support vector machines - a kernel approach , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[24]  Fernando Berzal,et al.  Data mining: concepts and techniques by Jiawei Han and Micheline Kamber , 2002, SGMD.

[25]  Robert P. W. Duin,et al.  Towards Scalable Prototype Selection by Genetic Algorithms with Fast Criteria , 2014, S+SSPR.

[26]  Pedro M. Domingos.  A few useful things to know about machine learning , 2012, Commun. ACM.

[27]  E. M. Wright,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[28]  Martin D. Levine,et al.  Feature extraction: A survey , 1969 .

[29]  Corinna Cortes,et al.  Boosting Decision Trees , 1995, NIPS.

[30]  Marcello Pelillo,et al.  Similarity-Based Pattern Analysis and Recognition , 2013, Advances in Computer Vision and Pattern Recognition.

[31]  Robert Sabourin,et al.  Dissimilarity Representation for Handwritten Signature Verification , 2013, AFHA.

[32]  Ana L. N. Fred,et al.  Learning Similarities from Examples Under the Evidence Accumulation Clustering Paradigm , 2013, Similarity-Based Pattern Analysis and Recognition.

[33]  Edwin R. Hancock,et al.  Spherical and Hyperbolic Embeddings of Data , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[35]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[36]  Antonio Torralba,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 80 Million Tiny Images: a Large Dataset for Non-parametric Object and Scene Recognition , 2022 .

[37]  Robert P. W. Duin,et al.  Prototype selection for dissimilarity-based classifiers , 2006, Pattern Recognit..

[38]  Igor Kononenko,et al.  Machine Learning and Data Mining: Introduction to Principles and Algorithms , 2007 .

[39]  Stephen K. Reed,et al.  Pattern recognition and categorization , 1972 .

[40]  Francisco Herrera,et al.  Prototype Selection for Nearest Neighbor Classification: Taxonomy and Empirical Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Dale Schuurmans,et al.  Boosting in the Limit: Maximizing the Margin of Learned Ensembles , 1998, AAAI/IAAI.

[42]  Yaokai Feng,et al.  Analyzing the Distribution of a Large-Scale Character Pattern Set Using Relative Neighborhood Graph , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[43]  Kaspar Riesen,et al.  Graph Classification and Clustering Based on Vector Space Embedding , 2010, Series in Machine Perception and Artificial Intelligence.

[44]  Volkmar Frinken,et al.  Tackling temporal pattern recognition by vector space embedding , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[45]  Michele Banko,et al.  Scaling to Very Very Large Corpora for Natural Language Disambiguation , 2001, ACL.

[46]  Antonio Torralba,et al.  LabelMe video: Building a video database with human annotations , 2009, 2009 IEEE 12th International Conference on Computer Vision.