Weakly Supervised Recognition of Daily Life Activities with Wearable Sensors

This paper considers scalable and unobtrusive activity recognition using on-body sensing for context awareness in wearable computing. Common methods for activity recognition rely on supervised learning requiring substantial amounts of labeled training data. Obtaining accurate and detailed annotations of activities is challenging, preventing the applicability of these approaches in real-world settings. This paper proposes new annotation strategies that substantially reduce the required amount of annotation. We explore two learning schemes for activity recognition that effectively leverage such sparsely labeled data together with more easily obtainable unlabeled data. Experimental results on two public data sets indicate that both approaches obtain results close to fully supervised techniques. The proposed methods are robust to the presence of erroneous labels occurring in real-world annotation data.

[1]  Maryam Mahdaviani,et al.  Fast and Scalable Training of Semi-Supervised CRFs with Application to Activity Recognition , 2007, NIPS.

[2]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[3]  Anthony Rowe,et al.  eWatch: a wearable sensor and notification platform , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[4]  Blake Hannaford,et al.  A Hybrid Discriminative/Generative Approach for Modeling Human Activities , 2005, IJCAI.

[5]  Philippe Golle,et al.  On using existing time-use study data for ubiquitous computing applications , 2008, UbiComp.

[6]  Paul Lukowicz,et al.  Using a complex multi-modal on-body sensor system for activity spotting , 2008, 2008 12th IEEE International Symposium on Wearable Computers.

[7]  Donghai Guan,et al.  Activity Recognition Based on Semi-supervised Learning , 2007, 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA 2007).

[8]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[9]  Bernt Schiele,et al.  Using rhythm awareness in long-term activity recognition , 2008, 2008 12th IEEE International Symposium on Wearable Computers.

[10]  Juan Carlos Niebles,et al.  A Hierarchical Model of Shape and Appearance for Human Action Classification , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Jesse Hoey,et al.  Semi-supervised learning of a POMDP model of Patient-Caregiver Interactions , 2005 .

[12]  Jeff A. Bilmes,et al.  Recognizing Activities and Spatial Context Using Wearable Sensors , 2006, UAI.

[13]  Michael J. Black,et al.  Parameterized Modeling and Recognition of Activities , 1999, Comput. Vis. Image Underst..

[14]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[15]  Liang Wang,et al.  Recognizing Human Activities from Silhouettes: Motion Subspace and Factorial Discriminative Graphical Model , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Gaetano Borriello,et al.  A Practical Approach to Recognizing Physical Activities , 2006, Pervasive.

[17]  Zhi-Hua Zhou Multi-Instance Learning : A Survey , 2004 .

[18]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[19]  Eric Horvitz,et al.  Experience sampling for building predictive user models: a comparative study , 2008, CHI.

[20]  Svetha Venkatesh,et al.  Activity recognition and abnormality detection with the switching hidden semi-Markov model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[21]  Thomas Hofmann,et al.  Support Vector Machines for Multiple-Instance Learning , 2002, NIPS.

[22]  Bernt Schiele,et al.  Discovery of activity patterns using topic models , 2008 .

[23]  Matthai Philipose,et al.  Common Sense Based Joint Training of Human Activity Recognizers , 2007, IJCAI.

[24]  Bernt Schiele,et al.  Multi-graph Based Semi-supervised Learning for Activity Recognition , 2009, 2009 International Symposium on Wearable Computers.

[25]  Maria-Florina Balcan,et al.  Person Identification in Webcam Images: An Application of Semi-Supervised Learning , 2005 .

[26]  Jennifer Healey,et al.  A Long-Term Evaluation of Sensing Modalities for Activity Recognition , 2007, UbiComp.

[27]  Bernt Schiele,et al.  Activity Recognition from Sparsely Labeled Data Using Multi-Instance Learning , 2009, LoCA.

[28]  Daniel P. Siewiorek,et al.  Activity recognition and monitoring using multiple sensors on different body positions , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[29]  Ronen Basri,et al.  Actions as space-time shapes , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[30]  Bernt Schiele,et al.  Exploring semi-supervised and active learning for activity recognition , 2008, 2008 12th IEEE International Symposium on Wearable Computers.

[31]  James A. Landay,et al.  MyExperience: a system for in situ tracing and capturing of user feedback on mobile phones , 2007, MobiSys '07.

[32]  Emmanuel Munguia Tapia,et al.  Toward Scalable Activity Recognition for Sensor Networks , 2006, LoCA.

[33]  Michael L. Littman,et al.  Activity Recognition from Accelerometer Data , 2005, AAAI.

[34]  Eric Horvitz,et al.  On Discarding, Caching, and Recalling Samples in Active Learning , 2007, UAI.

[35]  Ivan Laptev,et al.  On Space-Time Interest Points , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[36]  Ling Bao,et al.  A context-aware experience sampling tool , 2003, CHI Extended Abstracts.

[37]  Pietro Perona,et al.  Hybrid models for human motion recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[38]  Matthai Philipose,et al.  Unsupervised Activity Recognition Using Automatically Mined Common Sense , 2005, AAAI.

[39]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[40]  Bernt Schiele,et al.  Scalable Recognition of Daily Activities with Wearable Sensors , 2007, LoCA.

[41]  Paul Lukowicz,et al.  Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.