UvA-DARE ( Digital Academic Repository ) Activity recognition using semi-Markov models on real world smart home datasets

Accurately recognizing human activities from sensor data recorded in a smart home setting is a challenging task. Typically, probabilistic models such as the hidden Markov model (HMM) or conditional random fields (CRF) are used to map the observed sensor data onto the hidden activity states. A weakness of these models, however, is that the type of distribution used to model state durations is fixed. Hidden semi-Markov models (HSMM) and semi-Markov conditional random fields (SMCRF) model duration explicitly, allowing state durations to be modelled accurately. In this paper we compare the recognition performance of these models on multiple fully annotated real world datasets consisting of several weeks of data. In our experiments the HSMM consistently outperforms the HMM, showing that accurate duration modelling can result in a significant increase in recognition performance. SMCRFs only slightly outperform CRFs, showing that CRFs are more robust in dealing with violations of the modelling assumptions. The datasets used in our experiments are made available to the community to allow further experimentation.

[1]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[2]  Gregory D. Abowd,et al.  The Aware Home: A living laboratory for technologies for successful aging , 2002 .

[3]  Manuela M. Veloso,et al.  Conditional random fields for activity recognition , 2007, AAMAS '07.

[4]  Hanna M. Wallach,et al.  Efficient Training of Conditional Random Fields , 2002 .

[5]  Jeff A. Bilmes,et al.  Hierarchical Models for Activity Recognition , 2006, 2006 IEEE Workshop on Multimedia Signal Processing.

[6]  William W. Cohen,et al.  Semi-Markov Conditional Random Fields for Information Extraction , 2004, NIPS.

[7]  Jennifer Healey,et al.  A Long-Term Evaluation of Sensing Modalities for Activity Recognition , 2007, UbiComp.

[8]  Sajal K. Das,et al.  Smart Environments: Technology, Protocols and Applications (Wiley Series on Parallel and Distributed Computing) , 2004 .

[9]  Eric Horvitz,et al.  Layered representations for learning and inferring office activity from multiple sensory channels , 2004, Comput. Vis. Image Underst..

[10]  Christopher G. Atkeson,et al.  Assistive intelligent environments for automatic health monitoring , 2005 .

[11]  M. Ogawa,et al.  Analysis of activities of daily living in elderly people living alone: single-subject feasibility study. , 2004, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[12]  Kent Larson,et al.  The Design of a Portable Kit of Wireless Sensors for Naturalistic Data Collection , 2006, Pervasive.

[13]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[14]  Christopher G. Atkeson,et al.  Simultaneous Tracking and Activity Recognition (STAR) Using Many Anonymous, Binary Sensors , 2005, Pervasive.

[15]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[16]  Henry A. Kautz,et al.  Fine-grained activity recognition by aggregating abstract object usage , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[17]  Diane J. Cook,et al.  Smart environments - technology, protocols and applications , 2004 .

[18]  Ben Taskar,et al.  Introduction to statistical relational learning , 2007 .

[19]  Ramakant Nevatia,et al.  Coupled Hidden Semi Markov Models for Activity Recognition , 2007, 2007 IEEE Workshop on Motion and Video Computing (WMVC'07).

[20]  Fernando Pereira,et al.  Shallow Parsing with Conditional Random Fields , 2003, NAACL.

[21]  Svetha Venkatesh,et al.  Efficient duration and hierarchical modeling for human activity recognition , 2009, Artif. Intell..

[22]  Kevin P. Murphy Hidden semi-Markov models ( HSMMs ) , 2002 .

[23]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[24]  Einat Marhasev,et al.  Non-stationary Hidden Semi Markov Models in Activity Recognition , 2006 .

[25]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[26]  Juan Carlos Augusto,et al.  Designing Smart Homes, The Role of Artificial Intelligence , 2006, Designing Smart Homes.

[27]  Gwenn Englebienne,et al.  Accurate activity recognition in a home setting , 2008, UbiComp.

[28]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.