Complex activity recognition using context-driven activity theory and activity signatures

In pervasive and ubiquitous computing systems, human activity recognition has immense potential in a large number of application domains. Current activity recognition techniques (i) do not handle variations in sequence, concurrency and interleaving of complex activities; (ii) do not incorporate context; and (iii) require large amounts of training data. There is a lack of a unifying theoretical framework which exploits both domain knowledge and data-driven observations to infer complex activities. In this article, we propose, develop and validate a novel Context-Driven Activity Theory (CDAT) for recognizing complex activities. We develop a mechanism using probabilistic and Markov chain analysis to discover complex activity signatures and generate complex activity definitions. We also develop a Complex Activity Recognition (CAR) algorithm. It achieves an overall accuracy of 95.73% using extensive experimentation with real-life test data. CDAT utilizes context and links complex activities to situations, which reduces inference time by 32.5% and also reduces training data by 66%.

[1]  Nigel Davies,et al.  Structural Learning of Activities from Sparse Datasets , 2007, Fifth Annual IEEE International Conference on Pervasive Computing and Communications (PerCom'07).

[2]  Claudio Bettini,et al.  COSAR: hybrid reasoning for context-aware activity recognition , 2011, Personal and Ubiquitous Computing.

[3]  Arkady B. Zaslavsky,et al.  Towards a theory of context spaces , 2004, IEEE Annual Conference on Pervasive Computing and Communications Workshops, 2004. Proceedings of the Second.

[4]  Henry A. Kautz,et al.  Training Conditional Random Fields Using Virtual Evidence Boosting , 2007, IJCAI.

[5]  D. Cook,et al.  Author's Personal Copy Pervasive and Mobile Computing Activity Knowledge Transfer in Smart Environments , 2022 .

[6]  K. Gegenfurtner,et al.  Design Issues in Gaze Guidance Under review with ACM Transactions on Computer Human Interaction , 2009 .

[7]  Claudio Bettini,et al.  Context-Aware Activity Recognition through a Combination of Ontological and Statistical Reasoning , 2009, UIC.

[8]  Irfan A. Essa,et al.  Structure from Statistics - Unsupervised Activity Analysis using Suffix Trees , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  Daniel P. Siewiorek,et al.  Activity-Based Computing , 2008, IEEE Pervasive Computing.

[10]  Sung-Bae Cho,et al.  Activity Recognition Using Hierarchical Hidden Markov Models on a Smartphone with 3D Accelerometer , 2011, HAIS.

[11]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[12]  David Wasserman.  The Activity Checklist : A Tool for Representing the “ Space ” of Context , 1999 .

[13]  Heiner Stuckenschmidt,et al.  Recognizing interleaved and concurrent activities using qualitative and quantitative temporal relationships , 2011, Pervasive Mob. Comput..

[14]  R. Abelson.  Psychological status of the script concept. , 1981 .

[15]  Kristof Van Laerhoven.  Combining the Self-Organizing Map and K-Means Clustering for On-Line Classification of Sensor Data , 2001, ICANN.

[16]  J. Hsu,et al.  Joint Recognition of Multiple Concurrent Activities using Factorial Conditional Random Fields , 2007 .

[17]  Matthai Philipose,et al.  Common Sense Based Joint Training of Human Activity Recognizers , 2007, IJCAI.

[18]  Bernt Schiele,et al.  ADL recognition based on the combination of RFID and accelerometer sensing , 2008, 2008 Second International Conference on Pervasive Computing Technologies for Healthcare.

[19]  Arkady B. Zaslavsky,et al.  Multiple-Agent Perspectives in Reasoning About Situations for Context-Aware Pervasive Computing Systems , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[20]  Archan Misra,et al.  HARMONI: Context-aware Filtering of Sensor Data for Continuous Remote Health Monitoring , 2008, 2008 Sixth Annual IEEE International Conference on Pervasive Computing and Communications (PerCom).

[21]  Etienne Pardoux,et al.  Markov Processes and Applications: Algorithms, Networks, Genome and Finance , 2009 .

[22]  Chris D. Nugent,et al.  Evidential fusion of sensor data for activity recognition in smart homes , 2009, Pervasive Mob. Comput..

[23]  Jake K. Aggarwal,et al.  Recognition of Composite Human Activities through Context-Free Grammar Based Representation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  Guang-Zhong Yang,et al.  Body sensor networks , 2006 .

[25]  Paul Lukowicz,et al.  Recognizing Workshop Activity Using Body Worn Microphones and Accelerometers , 2004, Pervasive.

[26]  Shumin Zhai,et al.  A comparative evaluation of finger and pen stroke gestures , 2012, CHI.

[27]  Simon A. Dobson,et al.  Using situation lattices in sensor analysis , 2009, 2009 IEEE International Conference on Pervasive Computing and Communications.

[28]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[29]  Gerhard Tröster,et al.  Gestures are strings: efficient online gesture spotting and classification using string matching , 2007, BODYNETS.

[30]  Shumin Zhai,et al.  Introduction to sensing-based interaction , 2005, TCHI.

[31]  Gwenn Englebienne,et al.  Accurate activity recognition in a home setting , 2008, UbiComp.

[32]  Chris D. Nugent,et al.  A Knowledge-Driven Approach to Activity Recognition in Smart Homes , 2012, IEEE Transactions on Knowledge and Data Engineering.

[33]  Andry Rakotonirainy,et al.  A Survey of Research on Context-Aware Homes , 2003, ACSW.

[34]  Victor Kaptelinin,et al.  Methods & tools: The activity checklist: a tool for representing the “space” of context , 1999, INTR.

[35]  Frank Bomarius,et al.  An Event-Driven Approach to Activity Recognition in Ambient Assisted Living , 2009, AmI.

[36]  James A. Landay,et al.  The Mobile Sensing Platform: An Embedded Activity Recognition System , 2008, IEEE Pervasive Computing.

[37]  Yiannis Aloimonos,et al.  A Language for Human Action , 2007, Computer.

[38]  Paul Lukowicz,et al.  Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Gerhard Tröster,et al.  Recognition of dietary activity events using on-body sensors , 2008, Artif. Intell. Medicine.

[40]  Irfan A. Essa,et al.  Discovering Characteristic Actions from On-Body Sensor Data , 2006, 2006 10th IEEE International Symposium on Wearable Computers.

[41]  Arkady B. Zaslavsky,et al.  Complex Activity Recognition Using Context Driven Activity Theory in Home Environments , 2011, NEW2AN.

[42]  Jodi Forlizzi,et al.  Using context to reveal factors that affect physical activity , 2012, TCHI.

[43]  Heiner Stuckenschmidt,et al.  Recognizing interleaved and concurrent activities: A statistical-relational approach , 2011, 2011 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[44]  Kristof Van Laerhoven,et al.  What shall we teach our pants? , 2000, Digest of Papers. Fourth International Symposium on Wearable Computers.

[45]  Oliver Brdiczka,et al.  Learning Situation Models for Providing Context-Aware Services , 2007, HCI.

[46]  Jiangwen Deng,et al.  An HMM-based approach for gesture segmentation and recognition , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[47]  Henry A. Kautz,et al.  Inferring activities from interactions with objects , 2004, IEEE Pervasive Computing.

[48]  Lawrence B. Holder,et al.  Discovering Activities to Recognize and Track in a Smart Environment , 2011, IEEE Transactions on Knowledge and Data Engineering.

[49]  Emil Jovanov,et al.  Stress monitoring using a distributed wireless intelligent sensor system. , 2003, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[50]  Ling Bao,et al.  Physical activity recognition from acceleration data under semi-naturalistic conditions , 2003 .

[51]  G. Englebienne,et al.  Transferring Knowledge of Activity Recognition across Sensor Networks , 2010, Pervasive.

[52]  ZaslavskyArkady,et al.  Complex activity recognition using context-driven activity theory and activity signatures , 2013 .

[53]  Oliver Brdiczka,et al.  Temporal task footprinting: identifying routine tasks by their temporal patterns , 2010, IUI '10.

[54]  Gregory D. Abowd,et al.  Charting past, present, and future research in ubiquitous computing , 2000, TCHI.

[55]  Qiang Yang,et al.  CIGAR: Concurrent and Interleaving Goal and Activity Recognition , 2008 .

[56]  David W. McDonald,et al.  Activity sensing in the wild: a field trial of ubifit garden , 2008, CHI.

[57]  Jian Lu,et al.  epSICAR: An Emerging Patterns based approach to sequential, interleaved and Concurrent Activity Recognition , 2009, 2009 IEEE International Conference on Pervasive Computing and Communications.

[58]  Gregory D. Abowd,et al.  Providing architectural support for building context-aware applications , 2000 .

[59]  Eric Horvitz,et al.  Layered representations for human activity recognition , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[60]  Guang-Zhong Yang,et al.  Bioinspired Design for Body Sensor Networks [Life Sciences] , 2013, IEEE Signal Processing Magazine.

[61]  Jesús Favela,et al.  Activity-Aware Computing for Healthcare , 2008, IEEE Pervasive Computing.

[62]  Thomas Stiefmeier,et al.  Real-time spotting of human activities in industrial environments , 2008 .

[63]  Paul Lukowicz,et al.  Wearable Activity Tracking in Car Manufacturing , 2008, IEEE Pervasive Computing.

[64]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[65]  Juan Ye,et al.  Using Situation Lattices to Model and Reason about Context , 2007 .

[66]  Bernt Schiele,et al.  Unsupervised Discovery of Structure in Activity Data Using Multiple Eigenspaces , 2006, LoCA.