Wearable sensor activity analysis using semi-Markov models with a grammar

Detailed monitoring of training sessions of elite athletes is an important component of their training. In this paper we describe an application that performs a precise segmentation and labeling of swimming sessions. This allows a comprehensive breakdown of the training session, including lap times, detailed statistics of strokes, and turns. To this end we use semi-Markov models (SMM), a formalism for labeling and segmenting sequential data, trained in a max-margin setting. To reduce the computational complexity of the task and at the same time enforce sensible output, we introduce a grammar into the SMM framework. Using the trained model on test swimming sessions of different swimmers provides highly accurate segmentation as well as perfect labeling of individual segments. The results are significantly better than those achieved by discriminative hidden Markov models.

[1]  Sadaoki Furui,et al.  Speaker-independent isolated word recognition using dynamic features of speech spectrum , 1986, IEEE Trans. Acoust. Speech Signal Process..

[2]  Michael L. Littman,et al.  Activity Recognition from Accelerometer Data , 2005, AAAI.

[3]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[4]  Brian T. Smith,et al.  Application of a neuro-fuzzy network for gait event detection using electromyography in the child with Cerebral palsy , 2005, IEEE Transactions on Biomedical Engineering.

[5]  Thomas Hofmann,et al.  Large Scale Hidden Semi-Markov SVMs , 2007 .

[6]  Ben Taskar,et al.  Max-Margin Markov Networks , 2003, NIPS.

[7]  B.T. Smith,et al.  Evaluation of force-sensing resistors for gait event detection to trigger electrical stimulation to improve walking in the child with cerebral palsy , 2002, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  D. Bertsekas,et al.  Convergen e Rate of In remental Subgradient Algorithms , 2000 .

[9]  R Williamson,et al.  Gait event detection for FES using accelerometers and supervised machine learning. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[10]  Yangsheng Xu,et al.  Gait Event Detection with Intelligent Shoes , 2007, 2007 International Conference on Information Acquisition.

[11]  Joseph A. Paradiso,et al.  A compact, wireless, wearable sensor network for interactive dance ensembles , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[12]  Ilkka Korhonen,et al.  Detection of Daily Activities and Sports With Wearable Sensors in Controlled and Uncontrolled Conditions , 2008, IEEE Transactions on Information Technology in Biomedicine.

[13]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[14]  Thomas Hofmann,et al.  Large Margin Methods for Structured and Interdependent Output Variables , 2005, J. Mach. Learn. Res..

[15]  N. Limnios,et al.  Semi-Markov Models and Applications , 2011 .

[16]  Fabrice Lefèvre,et al.  On the influence of the delta coefficients in a HMM-based speech recognition system , 1998, ICSLP.

[17]  Ronen Basri,et al.  Actions as Space-Time Shapes , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Bill N. Schilit,et al.  Context-aware computing applications , 1994, Workshop on Mobile Computing Systems and Applications.

[19]  Alexander J. Smola,et al.  Semi-Markov Models for Sequence Segmentation , 2007, EMNLP.

[20]  Michael I. Jordan,et al.  An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators , 2008, ICML '08.

[21]  D.A. James,et al.  An accelerometer based sensor platform for insitu elite athlete performance analysis , 2004, Proceedings of IEEE Sensors, 2004..

[22]  S. Uryasev,et al.  Stochastic optimization : Algorithms and Applications , 2001 .

[23]  Paolo Bonato,et al.  Analysis of the severity of dyskinesia in patients with Parkinson's disease via wearable sensors , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[24]  William W. Cohen,et al.  Semi-Markov Conditional Random Fields for Information Extraction , 2004, NIPS.

[25]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[26]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[27]  M.M. Skelly,et al.  Real-time gait event detection for paraplegic FES walking , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  Guang-Zhong Yang,et al.  Pervasive body sensor network: an approach to monitoring the post-operative surgical patient , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[29]  Osama Masoud,et al.  Recognizing human activities , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[30]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[31]  T. Sinkjaer,et al.  Evaluating robustness of gait event detection based on machine learning and natural sensors , 2004, IEEE Transactions on Neural Systems and Rehabilitation Engineering.