Searching for variable-speed motions in long sequences of motion capture data

Abstract Motion capture data digitally represent human movements by sequences of body configurations in time. Subsequence searching in long sequences of such spatio-temporal data is difficult as query-relevant motions can vary in execution speeds and styles and can occur anywhere in a very long data sequence. To deal with these problems, we employ a fast and effective similarity measure that is elastic. The property of elasticity enables matching of two overlapping but slightly misaligned subsequences with a high confidence. Based on the elasticity, the long data sequence is partitioned into overlapping segments that are organized in multiple levels. The number of levels and sizes of overlaps are optimized to generate a modest number of segments while being able to trace an arbitrary query. In a retrieval phase, a query is always represented as a single segment and fast matched against segments within a relevant level without any costly post-processing. Moreover, visiting adjacent levels makes possible subsequence searching of time-warped (i.e., faster or slower executed) queries. To efficiently search on a large scale, segment features can be binarized and segmentation levels independently indexed. We experimentally demonstrate effectiveness and efficiency of the proposed approach for subsequence searching on a real-life dataset.

[1]  Zhaoqi Wang,et al.  Indexing and retrieval of human motion data by a hierarchical tree , 2009, VRST '09.

[2]  Tobias Schreck,et al.  MotionExplorer: Exploratory Search in Human Motion Capture Data Based on Hierarchical Aggregation , 2013, IEEE Transactions on Visualization and Computer Graphics.

[3]  Thomas Seidl,et al.  Spatiotemporal Similarity Search in 3D Motion Capture Gesture Streams , 2015, SSTD.

[4]  Michelle Karg,et al.  Movement Primitive Segmentation for Human Motion Modeling: A Framework for Analysis , 2016, IEEE Transactions on Human-Machine Systems.

[5]  Pavel Zezula,et al.  Fast Subsequence Matching in Motion Capture Data , 2017, ADBIS.

[6]  Michael Neff,et al.  Deep signatures for indexing and retrieval in large motion databases , 2015, MIG.

[7]  Jernej Barbic,et al.  Segmenting Motion Capture Data into Distinct Behaviors , 2004, Graphics Interface.

[8]  Pavel Zezula,et al.  Assessing similarity models for human‐motion retrieval applications , 2016, Comput. Animat. Virtual Worlds.

[9]  Christos Faloutsos,et al.  Fast subsequence matching in time-series databases , 1994, SIGMOD '94.

[10]  Yoon-Joon Lee,et al.  A filtering method for searching similar multidimensional sequences under the time-warping distance , 2003, Inf. Syst..

[11]  C.-C. Jay Kuo,et al.  Automatic Human Mocap Data Classification , 2014, IEEE Transactions on Multimedia.

[12]  Pavel Zezula,et al.  Motion Images: An Effective Representation of Motion Capture Data for Similarity Search , 2015, SISAP.

[13]  Pavel Zezula,et al.  Similarity Search - The Metric Space Approach , 2005, Advances in Database Systems.

[14]  Pavel Zezula,et al.  Similarity Searching in Long Sequences of Motion Capture Data , 2016, SISAP.

[15]  Tak-Chung Fu,et al.  A review on time series data mining , 2011, Eng. Appl. Artif. Intell..

[16]  Hans-Peter Seidel,et al.  Efficient and Robust Annotation of Motion Capture Data , 2009 .

[17]  Arno Zinke,et al.  Fast local and global similarity searches in large motion capture databases , 2010, SCA '10.

[18]  Yang-Sae Moon,et al.  Efficient time-series subsequence matching using duality in constructing windows , 2001, Inf. Syst..

[19]  Thierry Dutoit,et al.  3D skeleton‐based action recognition by representing motion capture sequences as 2D‐RGB images , 2017, Comput. Animat. Virtual Worlds.

[20]  Huaijiang Sun,et al.  Automated human motion segmentation via motion regularities , 2013, The Visual Computer.

[21]  Jessica K. Hodgins,et al.  Hierarchical Aligned Cluster Analysis for Temporal Clustering of Human Motion , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Zhigang Deng,et al.  Perceptually consistent example-based human motion retrieval , 2009, I3D '09.

[23]  David Novak,et al.  Efficient Image Search with Neural Net Features , 2015, SISAP.

[24]  Pavel Zezula,et al.  A Key-Pose Similarity Algorithm for Motion Data Retrieval , 2013, ACIVS.

[25]  Eamonn J. Keogh,et al.  Time series shapelets: a novel technique that allows accurate, interpretable and fast classification , 2010, Data Mining and Knowledge Discovery.

[26]  Chao-Hung Lin,et al.  Human Motion Retrieval from Hand-Drawn Sketch , 2012, IEEE Transactions on Visualization and Computer Graphics.

[27]  Zhaoqi Wang,et al.  Efficient motion data indexing and retrieval with local similarity measure of motion strings , 2009, The Visual Computer.

[28]  Kyoji Kawagoe,et al.  A coarse-to-fine method for subsequence matching of human behavior using multi-dimensional time-series approximation , 2015, iiWAS.

[29]  Atsushi Nakazawa,et al.  A puppet interface for retrieval of motion capture data , 2011, SCA '11.

[30]  Howard Leung,et al.  Retrieval of logically relevant 3D human motions by Adaptive Feature Selection with Graded Relevance Feedback , 2012, Pattern Recognit. Lett..

[31]  Norman I. Badler,et al.  Efficient motion retrieval in large motion databases , 2013, I3D '13.

[32]  Pavel Zezula,et al.  Effective and efficient similarity searching in motion capture data , 2017, Multimedia Tools and Applications.

[33]  Norman I. Badler,et al.  Semantic Segmentation of Motion Capture Using Laban Movement Analysis , 2007, IVA.

[34]  Tido Röder,et al.  Efficient content-based retrieval of motion capture data , 2005, SIGGRAPH 2005.

[35]  Luc Van Gool,et al.  Deep Learning on Lie Groups for Skeleton-Based Action Recognition , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).