KCAR: A knowledge-driven approach for concurrent activity recognition

Abstract Recognising human activities from sensors embedded in an environment or worn on bodies is an important and challenging research topic in pervasive computing. Existing work on activity recognition is mainly concerned with identifying single user sequential activities from well-scripted or pre-segmented sequences of sensor events. However a real-world environment often contains multiple users, with each performing activities simultaneously, in their own way and with no explicit instructions to follow. Recognising multi-user concurrent activities is challenging, but essential for designing applications for real environments. This paper presents a novel Knowledge-driven approach for Concurrent Activity Recognition (KCAR). Within KCAR, we explore the semantics underlying each sensor event and use semantic dissimilarity to segment a continuous sensor sequence into fragments, each of which corresponds to one ongoing activity. We exploit the Pyramid Match Kernel, with a strength in approximate matching on hierarchical concepts, to recognise activities of varying grained constraints from a potentially noisy sensor sequence. We conduct an empirical evaluation on a large-scale real-world data set that is collected over one year and consists of 2.8 millions of sensor events. Our results demonstrate that KCAR achieves an average recognition accuracy of 91%.

[1]  Diane J Cook,et al.  Assessing the Quality of Activities in a Smart Environment , 2009, Methods of Information in Medicine.

[2]  Oliver Brdiczka,et al.  Detecting Individual Activities from Video in a Smart Home , 2007, KES.

[3]  Henry A. Kautz,et al.  Improving the recognition of interleaved activities , 2008, UbiComp.

[4]  Simon A. Dobson,et al.  A top-level ontology for smart environments , 2011, Pervasive Mob. Comput..

[5]  Gregory D. Abowd,et al.  The Aware Home: A Living Laboratory for Ubiquitous Computing Research , 1999, CoBuild.

[6]  Bernt Schiele,et al.  Scalable Recognition of Daily Activities with Wearable Sensors , 2007, LoCA.

[7]  Diane J Cook,et al.  How Smart Is Your Home? , 2012, Science.

[8]  Amit P. Sheth,et al.  Semantic Sensor Web , 2008, IEEE Internet Computing.

[9]  Diego Calvanese,et al.  The Description Logic Handbook: Theory, Implementation, and Applications , 2003, Description Logic Handbook.

[10]  Jesse Hoey,et al.  Sensor-Based Activity Recognition , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[11]  Svetha Venkatesh,et al.  Recognising Behaviours of Multiple People with Hierarchical Probabilistic Model and Statistical Data Association , 2006, BMVC.

[12]  Simon A. Dobson,et al.  Activity recognition using temporal evidence theory , 2010, J. Ambient Intell. Smart Environ..

[13]  Simon A. Dobson,et al.  Using situation lattices in sensor analysis , 2009, 2009 IEEE International Conference on Pervasive Computing and Communications.

[14]  Diane J. Cook,et al.  Mining Sensor Streams for Discovering Human Activity Patterns over Time , 2010, 2010 IEEE International Conference on Data Mining.

[15]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[16]  Heiner Stuckenschmidt,et al.  Recognizing interleaved and concurrent activities: A statistical-relational approach , 2011, 2011 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[17]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[18]  David W. Conrath,et al.  Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy , 1997, ROCLING/IJCLCLP.

[19]  Jennifer Healey,et al.  A Long-Term Evaluation of Sensing Modalities for Activity Recognition , 2007, UbiComp.

[20]  Daniel P. Siewiorek,et al.  Activity recognition and monitoring using multiple sensors on different body positions , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[21]  Tae-Kyun Kim,et al.  Real-time Action Recognition by Spatiotemporal Semantic and Structural Forests , 2010, BMVC.

[22]  Jesse Hoey,et al.  Activity Recognition in Pervasive Intelligent Environments , 2011 .

[23]  Daniel Borrajo,et al.  A dynamic sliding window approach for activity recognition , 2011, UMAP'11.

[24]  Shaogang Gong,et al.  Recognition of group activities using dynamic probabilistic networks , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[25]  Jian Lu,et al.  Mining Emerging Patterns for recognizing activities of multiple users in pervasive computing , 2009, 2009 6th Annual International Mobile and Ubiquitous Systems: Networking & Services, MobiQuitous.

[26]  Diane J. Cook,et al.  Author's Personal Copy Pervasive and Mobile Computing Ambient Intelligence: Technologies, Applications, and Opportunities , 2022 .

[27]  Simon A. Dobson,et al.  Representing and Manipulating Situation Hierarchies using Situation Lattices , 2008, Rev. d'Intelligence Artif..

[28]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[29]  Chris D. Nugent,et al.  Partitioning time series sensor data for activity recognition , 2009, 2009 9th International Conference on Information Technology and Applications in Biomedicine.

[30]  Gwenn Englebienne,et al.  Human activity recognition from wireless sensor network data: benchmark and software , 2011 .

[31]  Qiang Yang,et al.  CIGAR: Concurrent and Interleaving Goal and Activity Recognition , 2008 .

[32]  Henry A. Kautz,et al.  Fine-grained activity recognition by aggregating abstract object usage , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[33]  Claudio Bettini,et al.  Context-Aware Activity Recognition through a Combination of Ontological and Statistical Reasoning , 2009, UIC.

[34]  Trevor Darrell,et al.  The Pyramid Match Kernel: Efficient Learning with Sets of Features , 2007, J. Mach. Learn. Res..

[35]  Simon A. Dobson,et al.  Situation identification techniques in pervasive computing: A review , 2012, Pervasive Mob. Comput..

[36]  Matthai Philipose,et al.  Common Sense Based Joint Training of Human Activity Recognizers , 2007, IJCAI.

[37]  Liming Chen,et al.  Dynamic sensor data segmentation for real-time knowledge-driven activity recognition , 2014, Pervasive Mob. Comput..

[38]  Peter Dalgaard,et al.  Introductory statistics with R , 2002, Statistics and computing.

[39]  Gwenn Englebienne,et al.  Accurate activity recognition in a home setting , 2008, UbiComp.

[40]  Chris D. Nugent,et al.  A Knowledge-Driven Approach to Activity Recognition in Smart Homes , 2012, IEEE Transactions on Knowledge and Data Engineering.

[41]  Qiang Yang,et al.  Sensor-Based Abnormal Human-Activity Detection , 2008, IEEE Transactions on Knowledge and Data Engineering.

[42]  Jian Lu,et al.  An unsupervised approach to activity recognition and segmentation based on object-use fingerprints , 2010, Data Knowl. Eng..

[43]  Philip Resnik,et al.  Using Information Content to Evaluate Semantic Similarity in a Taxonomy , 1995, IJCAI.

[44]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[45]  Diane J. Cook,et al.  Activity recognition on streaming sensor data , 2014, Pervasive Mob. Comput..

[46]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.

[47]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  Diane J. Cook,et al.  Human Activity Recognition and Pattern Discovery , 2010, IEEE Pervasive Computing.

[49]  Arkady B. Zaslavsky,et al.  Recognizing Concurrent and Interleaved Activities in Social Interactions , 2011, 2011 IEEE Ninth International Conference on Dependable, Autonomic and Secure Computing.

[50]  George A. Miller,et al.  Using Corpus Statistics and WordNet Relations for Sense Identification , 1998, CL.