QuActive: A Quality of Activities Monitoring and Notification System

In order to notify users about potentially unsafe situations and to track mistakes or efficiency performing activities, it is important to monitor the quality of performing an activity and identify the missing/wrong steps. However, the state-of-the-art activity recognition frameworks ignore such details and impose constraints on sensor values, the types of detected activities (no parallel/interleaved/joint activities), or the number of users, which reduce the robustness of the system in the real world settings. Therefore, we present QuActive, a grammar based general purpose framework for modeling activities and micro-activities that retains the details of the activity steps, quantifies activity quality, and notifies users about missing steps and unsafe situations. In order to show the versatility of QuActive, we evaluate the framework on three different public datasets that have interleaved activities, parallel and co-operative activities, and activities of cognitively declined patients with quality information labeled. In all cases, QuActive outperforms the state-of-the-art techniques applied on these data sets. In addition, we have deployed the system in a real home and collected data in a semi-controlled setting to evaluate the performance of the system in real settings. The results show that QuActive recognizes more than 90% of the defined micro-activities and the grammar detects almost all the defined activities from the recognized micro-activities.

[1]  Archan Misra,et al.  CACE: Exploiting Behavioral Interactions for Improved Activity Recognition in Multi-inhabitant Smart Homes , 2016, 2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS).

[2]  Claudio Bettini,et al.  Fine-grained recognition of abnormal behaviors for early detection of mild cognitive impairment , 2015, 2015 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[3]  Dieter Fox,et al.  Fine-grained kitchen activity recognition using RGB-D , 2012, UbiComp.

[4]  Qiang Li,et al.  Grammar-based, posture- and context-cognitive detection for falls with different activity levels , 2011, Wireless Health.

[5]  Gopal Gupta,et al.  Timed Definite Clause Omega-Grammars , 2010, ICLP.

[6]  Henry A. Kautz,et al.  Fine-grained activity recognition by aggregating abstract object usage , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[7]  Christopher G. Atkeson,et al.  Simultaneous Tracking and Activity Recognition (STAR) Using Many Anonymous, Binary Sensors , 2005, Pervasive.

[8]  Mohammed Feham,et al.  Multioccupant Activity Recognition in Pervasive Smart Home Environments , 2015, ACM Comput. Surv..

[9]  Max Van Kleek,et al.  A Practical Activity Capture Framework for Personal, Lifetime User Modeling , 2007, User Modeling.

[10]  Álvaro Marco,et al.  A Smart Kitchen for Ambient Assisted Living , 2014, Sensors.

[11]  Andreas Savvides,et al.  Extracting spatiotemporal human activity patterns in assisted living using a home sensor network , 2008, PETRA.

[12]  A. Savvides,et al.  A sensory grammar for inferring behaviors in sensor networks , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[13]  Patrick Pérez,et al.  View-Independent Action Recognition from Temporal Self-Similarities , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  John A. Stankovic,et al.  MedRem: an interactive medication reminder and tracking system on wrist devices , 2016, 2016 IEEE Wireless Health (WH).

[15]  Irfan A. Essa,et al.  Recognizing multitasked activities from video using stochastic context-free grammar , 2002, AAAI/IAAI.

[16]  Juan Carlos Niebles,et al.  Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification , 2010, ECCV.

[17]  Constantine Stephanidis,et al.  Universal access in the information society , 1999, HCI.

[18]  Nadir Weibel,et al.  ChronoViz: a system for supporting navigation of time-coded data , 2011, CHI Extended Abstracts.

[19]  Mark Johnson,et al.  Probabilistic Grammars and their Applications , 2015 .

[20]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[21]  Diane J Cook,et al.  Tracking Activities in Complex Settings Using Smart Environment Technologies. , 2009, International journal of biosciences, psychiatry, and technology.

[22]  Jake K. Aggarwal,et al.  Recognition of Composite Human Activities through Context-Free Grammar Based Representation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[23]  John A. Stankovic,et al.  SARRIMA: smart ADL recognizer and resident identifier in multi-resident accommodations , 2015, Wireless Health.

[24]  Andreas Savvides,et al.  Extracting spatiotemporal human activity patterns in assisted living using a home sensor network , 2008, PETRA '08.

[25]  Bernt Schiele,et al.  A database for fine grained activity detection of cooking activities , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Diane J. Cook,et al.  Recognizing independent and joint activities among multiple residents in smart environments , 2010, J. Ambient Intell. Humaniz. Comput..

[27]  Prafulla N. Dawadi,et al.  An approach to cognitive assessment in smart home , 2011, DMMH '11.