E ± cient Query Processing in 3 D Motion Capture Gesture Databases

One of the most fundamental challenges when accessing gestural patterns in 3D motion capture databases is the de ̄nition of spatiotemporal similarity. While distance-based similarity models such as the Gesture Matching Distance on gesture signatures are able to leverage the spatial and temporal characteristics of gestural patterns, their applicability to large 3D motion capture databases is limited due to their high computational complexity. To this end, we present a lower bound approximation of the Gesture Matching Distance that can be utilized in an optimal multi-step query processing architecture in order to support e±cient query processing. We investigate the performance in terms of accuracy and e±ciency based on 3D motion capture databases and show that our approach is able to achieve an increase in e±ciency of more than one order of magnitude with a negligible loss in accuracy. In addition, we discuss di®erent applications in the digital humanities in order to highlight the signi ̄cance of similarity search approaches in the research ̄eld of gestural pattern analysis.

[1]  R. Watson.  A Survey of Gesture Recognition Techniques , 1993 .

[2]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  KwangYun Wohn,et al.  Recognition of hand gestures with 3D, nonlinear arm movement , 1997, Pattern Recognit. Lett..

[4]  Hans-Peter Kriegel,et al.  Optimal multi-step k-nearest neighbor search , 1998, SIGMOD '98.

[5]  Ying Wu,et al.  Vision-Based Gesture Recognition: A Review , 1999, Gesture Workshop.

[6]  Jr. Joseph J. LaViola,et al.  A Survey of Hand Posture and Gesture Recognition Techniques and Technology , 1999 .

[7]  John Lyons,et al.  语义学引论 = Linguistic Semantics , 2000 .

[8]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[9]  S. Gong,et al.  Recognition of human gestures and behaviour based on motion trajectories , 2002, Image Vis. Comput..

[10]  Dimitrios Gunopulos,et al.  Indexing multi-dimensional time-series with support for multiple distance measures , 2003, KDD '03.

[11]  H. H. Clark.  Pointing and placing. , 2003 .

[12]  Cem Keskin,et al.  REAL TIME HAND TRACKING AND 3D GESTURE RECOGNITION FOR INTERACTIVE INTERFACES USING HMM , 2003 .

[13]  Eamonn J. Keogh,et al.  Exact indexing of dynamic time warping , 2002, Knowledge and Information Systems.

[14]  Lei Chen,et al.  On The Marriage of Lp-norms and Edit Distance , 2004, VLDB.

[15]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[16]  R. Gibbs.  Embodiment and Cognitive Science: Concepts , 2005 .

[17]  Qiang Wang,et al.  Elastic Partial Matching of Time Series , 2005, PKDD.

[18]  Dimitrios Gunopulos,et al.  Elastic Translation Invariant Matching of Trajectories , 2005, Machine Learning.

[19]  Lei Chen,et al.  Robust and fast similarity search for moving object trajectories , 2005, SIGMOD '05.

[20]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[21]  A. Goldberg.  Constructions at Work: The Nature of Generalization in Language , 2006 .

[22]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[23]  Eraldo Ribeiro,et al.  Human Motion Recognition Using Isomap and Dynamic Time Warping , 2007, Workshop on Human Motion.

[24]  Seong-Whan Lee,et al.  Recognizing hand gestures using dynamic Bayesian network , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[25]  Sang Uk Lee,et al.  Color-Based Image Retrieval Using Perceptually Modified Hausdorff Distance , 2008, EURASIP J. Image Video Process..

[26]  Kenny R. Coventry,et al.  Spatial Language and Dialogue , 2009, Explorations in language and space.

[27]  Hsiao-Lung Chan,et al.  Human identification by quantifying similarity and dissimilarity in electrocardiogram phase space , 2009, Pattern Recognit..

[28]  Jianyu Yang,et al.  A new descriptor for 3D trajectory recognition via modified CDTW , 2010, 2010 IEEE International Conference on Automation and Logistics.

[29]  Thomas Seidl,et al.  A comparative study of similarity measures for content-based multimedia retrieval , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[30]  Manolya Kavakli,et al.  A survey of speech-hand gesture recognition for the development of multimodal interfaces in computer games , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[31]  B. Bergen,et al.  Grammatical aspect and mental simulation , 2010, Brain and Language.

[32]  Thomas Seidl,et al.  Signature Quadratic Form Distance , 2010, CIVR '10.

[33]  Heung-Il Suk,et al.  Hand gesture recognition based on dynamic Bayesian network framework , 2010, Pattern Recognit..

[34]  Thomas Seidl,et al.  Towards a Mobile Health Context Prediction: Sequential Pattern Mining in Multiple Streams , 2011, 2011 IEEE 12th International Conference on Mobile Data Management.

[35]  Rafiqul Zaman Khan,et al.  Survey on Various Gesture Recognition Technologies and Techniques , 2012 .

[36]  Thomas Seidl,et al.  On Stability of Adaptive Similarity Measures for Content-Based Image Retrieval , 2012, MMM.

[37]  S. Abdul-Kareem,et al.  Static hand gesture recognition using neural networks , 2012 .

[38]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[39]  Stefan Kopp,et al.  Gesture-based Object Recognition using Histograms of Guiding Strokes , 2012, BMVC.

[40]  程俊,et al.  Feature Fusion for 3D Hand Gesture Recognition by Learning a Shared Hidden Space , 2012 .

[41]  Irene Mittelberg,et al.  The exbodied mind: Cognitive-semiotic principles as motivating forces in gesture , 2013 .

[42]  David McNeill,et al.  Body – Language – Communication: An International Handbook on Multimodality in Human Interaction , 2013 .

[43]  Helman Stern,et al.  Most discriminating segment - Longest common subsequence (MDSLCS) algorithm for dynamic hand gesture classification , 2013, Pattern Recognit. Lett..

[44]  Jana Bressem,et al.  70. A linguistic perspective on the notation of form features in gestures , 2013 .

[45]  Guillaume Doisy,et al.  Position-invariant, real-time gesture recognition based on dynamic time warping , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[46]  Tarik Arici,et al.  Robust gesture recognition using feature pre-processing and weighted dynamic time warping , 2014, Multimedia Tools and Applications.

[47]  A. Cienki.  Image schemas and mimetic schemas in cognitive llinguistics and gesture studies , 2013 .

[48]  C. Debras.  L'expression multimodale du positionnement interactionnel (multimodal stance-taking) : étude d'un corpus oral vidéo de discussions sur l'environnement en anglais britannique , 2013 .

[49]  Christian Beecks,et al.  Distance based similarity models for content based multimedia retrieval , 2013 .

[50]  Thomas Seidl,et al.  Signature matching distance for content-based image retrieval , 2013, ICMR.

[51]  Elena Mugellini,et al.  A Survey of Datasets for Human Gesture Recognition , 2014, HCI.

[52]  Marwan Hassani,et al.  Efficient clustering of big data streams , 2015 .

[53]  Thomas Seidl,et al.  Spatiotemporal Similarity Search in 3D Motion Capture Gesture Streams , 2015, SSTD.

[54]  Thomas Seidl,et al.  Efficient Distance-Based Gestural Pattern Mining in Spatiotemporal 3D Motion Capture Databases , 2015, 2015 IEEE International Conference on Data Mining Workshop (ICDMW).

[55]  L. Faust,et al.  Aspect An Introduction To The Study Of Verbal Aspect And Related Problems , 2016 .

[56]  Steven Schoonjans,et al.  Modalpartikeln als multimodale Konstruktionen. Eine korpusbasierte Kookkurrenzanalyse von Modalpartikeln und Gestik im Deutschen. , 2018 .