Matching incomplete time series with dynamic time warping: an algorithm and an application to post-stroke rehabilitation

OBJECTIVE The purpose of this study was to assess the performance of a real-time ("open-end") version of the dynamic time warping (DTW) algorithm for the recognition of motor exercises. Given a possibly incomplete input stream of data and a reference time series, the open-end DTW algorithm computes both the size of the prefix of reference which is best matched by the input, and the dissimilarity between the matched portions. The algorithm was used to provide real-time feedback to neurological patients undergoing motor rehabilitation. METHODS AND MATERIALS We acquired a dataset of multivariate time series from a sensorized long-sleeve shirt which contains 29 strain sensors distributed on the upper limb. Seven typical rehabilitation exercises were recorded in several variations, both correctly and incorrectly executed, and at various speeds, totaling a data set of 840 time series. Nearest-neighbour classifiers were built according to the outputs of open-end DTW alignments and their global counterparts on exercise pairs. The classifiers were also tested on well-known public datasets from heterogeneous domains. RESULTS Nonparametric tests show that (1) on full time series the two algorithms achieve the same classification accuracy (p-value =0.32); (2) on partial time series, classifiers based on open-end DTW have a far higher accuracy (kappa=0.898 versus kappa=0.447;p<10(-5)); and (3) the prediction of the matched fraction follows closely the ground truth (root mean square <10%). The results hold for the motor rehabilitation and the other datasets tested, as well. CONCLUSIONS The open-end variant of the DTW algorithm is suitable for the classification of truncated quantitative time series, even in the presence of noise. Early recognition and accurate class prediction can be achieved, provided that enough variance is available over the time span of the reference. Therefore, the proposed technique expands the use of DTW to a wider range of applications, such as real-time biofeedback systems.

[1]  Danilo De Rossi,et al.  Electroactive polymer-based devices for e-textiles in biomedicine , 2005, IEEE Trans. Inf. Technol. Biomed..

[2]  Mohammad Ghavami,et al.  A Novel Blind Watermarking of ECG Signals on Medical Images Using EZW Algorithm , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[3]  Simon Dixon,et al.  An On-Line Time Warping Algorithm for Tracking Musical Performances , 2005, IJCAI.

[4]  David Burshtein,et al.  An enhanced dynamic time warping model for improved estimation of DTW parameters , 2003, IEEE Trans. Speech Audio Process..

[5]  Ryuichi Oka.  Spotting Method for Classification of Real World Data , 1998, Comput. J..

[6]  Li Wei,et al.  SAXually Explicit Images: Finding Unusual Shapes , 2006, Sixth International Conference on Data Mining (ICDM'06).

[7]  Abhinanda Sarkar,et al.  Shape Description by Time Series , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Ira J. Haimowitz,et al.  Managing temporal worlds for medical trend diagnosis , 1996, Artif. Intell. Medicine.

[9]  Elena Tsiporkova,et al.  Merging microarray cell synchronization experiments through curve alignment , 2007, Bioinform..

[10]  Ryo Kurazume,et al.  Early Recognition and Prediction of Gestures , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[11]  Aaron E. Rosenberg,et al.  Considerations in dynamic time warping algorithms for discrete word recognition , 1978 .

[12]  C. Posten,et al.  Supervision of bioprocesses using a dynamic time warping algorithm , 1996 .

[13]  Qiang Wang,et al.  An elastic partial shape matching technique , 2007, Pattern Recognit..

[14]  Eamonn J. Keogh,et al.  Exact indexing of dynamic time warping , 2002, Knowledge and Information Systems.

[15]  T. Nishimura,et al.  Spotting recognition of human gestures from time-varying images , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[16]  G. Lightbody,et al.  The effect of frequency band on quantitative EEG measures in neonates with Hypoxic-ischaemic encephalopathy , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[17]  Fei Wang,et al.  Shape-based Matching of ECG Recordings , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[18]  Aaron E. Rosenberg,et al.  Performance tradeoffs in dynamic time warping algorithms for isolated word recognition , 1980 .

[19]  Eamonn J. Keogh,et al.  Derivative Dynamic Time Warping , 2001, SDM.

[20]  S. Quaglini,et al.  Posture Classification via Wearable Strain Sensors for Neurological Rehabilitation , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[21]  P. Caselli,et al.  Classification of Motor Activities through Derivative Dynamic Time Warping applied on Accelerometer Data , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[22]  D. De Rossi,et al.  Wearable, redundant fabric-based sensor arrays for reconstruction of body segment posture , 2004, IEEE Sensors Journal.

[23]  Shunji Mori,et al.  A segmentation-free biometric writer verification method based on continuous dynamic programming , 2006, Pattern Recognit. Lett..

[24]  Yoonsik Tak,et al.  A Leaf Image Retrieval Scheme Based on Partial Dynamic Time Warping and Two-Level Filtering , 2007, 7th IEEE International Conference on Computer and Information Technology (CIT 2007).

[25]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[26]  Enzo Pasquale Scilingo,et al.  Strain-sensing fabrics for wearable kinaesthetic-like systems , 2003 .

[27]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[28]  Enzo Pasquale Scilingo,et al.  Strain sensing fabric for hand posture and gesture monitoring , 2005, IEEE Transactions on Information Technology in Biomedicine.

[29]  H. Sakoe,et al.  Two-level DP-matching--A dynamic programming-based pattern matching algorithm for connected word recognition , 1979 .

[30]  Paul R. Cohen,et al.  Using Dynamic Time Warping to Bootstrap HMM-Based Clustering of Time Series , 2001, Sequence Learning.

[31]  Steven Salzberg,et al.  On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach , 1997, Data Mining and Knowledge Discovery.

[32]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[33]  A. Lymberis,et al.  Guest Editorial New Generation of Smart Wearable Health Systems and Applications , 2005, EMBC 2005.

[34]  S. Henikoff,et al.  Amino acid substitution matrices from protein blocks. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Volkan Tuzcu,et al.  Dynamic time warping as a novel tool in pattern recognition of ECG changes in heart rhythm disturbances , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[36]  Catherine Garbay,et al.  Learning recurrent behaviors from heterogeneous multivariate time-series , 2007, Artif. Intell. Medicine.

[37]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[38]  Christos Faloutsos,et al.  Stream Monitoring under the Time Warping Distance , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[39]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[40]  S. Quaglini,et al.  A Multivariate Time-Warping Based Classifier for Gesture Recognition with Wearable Strain Sensors , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[41]  D. Inzitari,et al.  The Italian Guidelines for stroke prevention. The Stroke Prevention and Educational Awareness Diffusion (SPREAD) Collaboration. , 2000, Neurological sciences : official journal of the Italian Neurological Society and of the Italian Society of Clinical Neurophysiology.

[42]  Stan Salvador,et al.  FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space , 2004 .

[43]  Witold Kinsner,et al.  ECG frame classification using dynamic time warping , 2002, IEEE CCECE2002. Canadian Conference on Electrical and Computer Engineering. Conference Proceedings (Cat. No.02CH37373).

[44]  Toni Giorgino,et al.  Upper-limb rehabilitation exercises acquired through 29 elastomer strain sensors placed on fabric , 2008 .

[45]  Haiyuan Wu,et al.  Improvement of continuous dynamic programming for human gesture recognition , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[46]  Gerhard Tröster,et al.  Recognizing Upper Body Postures using Textile Strain Sensors , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[47]  Toni Giorgino,et al.  Remote Support for Stroke Rehabilitation: MyHeart's "Neurological Rehabilitation" Concept , 2007, MedInfo.

[48]  Peter Langhorne,et al.  Effects of Augmented Exercise Therapy Time After Stroke: A Meta-Analysis , 2004, Stroke.

[49]  Juan Carlos Augusto,et al.  Temporal reasoning for decision support in medicine , 2005, Artif. Intell. Medicine.

[50]  S. Quaglini,et al.  Economic benefit from clinical practice guideline compliance in stroke patient management. , 2004, Health policy.

[51]  T.O. Kvalseth,et al.  Measurement of interobserver agreement using a "standard": measure formulation and statistical inferences , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[52]  Cláudia Antunes,et al.  Temporal Data Mining: an overview , 2001 .

[53]  C. L. Giles,et al.  Sequence Learning - Paradigms, Algorithms, and Applications , 2001 .

[54]  R. Manmatha,et al.  Word image matching using dynamic time warping , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[55]  T. Warren Liao,et al.  Clustering of time series data - a survey , 2005, Pattern Recognit..

[56]  Michael J. Black,et al.  A Non-Parametric Bayesian Approach to Spike Sorting , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[57]  F. Itakura,et al.  Minimum prediction residual principle applied to speech recognition , 1975 .

[58]  Cory S Myers,et al.  A comparative study of several dynamic time warping algorithms for speech recognition , 1980 .

[59]  Gonzalo Navarro,et al.  A guided tour to approximate string matching , 2001, CSUR.

[60]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[61]  Ralph Niels,et al.  Dynamic time warping: a new method in the study of poor handwriting. , 2008, Human movement science.

[62]  Ira J. Haimowitz,et al.  Knowledge-based trend detection and diagnosis , 1994 .

[63]  Nicola Vanello,et al.  Combination of event-related potentials and functional magnetic resonance imaging during single-letter reading , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[64]  Marcos Faúndez-Zanuy,et al.  On-line signature recognition based on VQ-DTW , 2007, Pattern Recognit..

[65]  E. Troisi,et al.  Early versus delayed inpatient stroke rehabilitation: a matched comparison conducted in Italy. , 2000, Archives of physical medicine and rehabilitation.

[66]  George M. Church,et al.  Aligning gene expression time series with time warping algorithms , 2001, Bioinform..

[67]  Carolyn McGregor,et al.  Temporal abstraction in intelligent clinical data analysis: A survey , 2007, Artif. Intell. Medicine.