Agent‐oriented activity recognition in the event calculus: An application for diabetic patients

We present a knowledge representation framework on the basis of the Event Calculus that allows an agent to recognize complex activities from low‐level observations received by multiple sensors, reason about the life cycle of such activities, and take action to support their successful completion. Activities are multivalue fluents that change according to events that occur in the environment. The parameters of an activity consist of a unique label, a set of participants involved in the performing of the activity, and a unique goal associated with the activity revealing the activity's desired outcome. Our contribution is the identification of an activity life cycle describing how activities can be started, interrupted, suspended, resumed, or completed over time, as well as how these can be represented. The framework also specifies activity goals, their associated life cycle, and their relation with the activity life cycle. We provide the complete implementation of the framework, which includes an activity generator that automatically creates synthetic sensor data in the form of event streams that represent the everyday lifestyle of a type 1 diabetic patient. Moreover, we test the framework by generating very large activity streams that we use to evaluate the performance of the recognition capability and study its relative merits.

[1]  James F. Allen,et al.  Actions and Events in Interval Temporal Logic , 1994, J. Log. Comput..

[2]  Alfonso E. Romero,et al.  Recognising lifestyle activities of diabetic patients with a smartphone , 2015, 2015 9th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth).

[3]  Henry A. Kautz,et al.  Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields , 2007, Int. J. Robotics Res..

[4]  Ioannis N. Kouris,et al.  A comparative study of pattern recognition classifiers to predict physical activities using smartphones and wearable body sensors. , 2012, Technology and health care : official journal of the European Society for Engineering and Medicine.

[5]  Alexander Artikis,et al.  An Event Calculus for Event Recognition , 2015, IEEE Transactions on Knowledge and Data Engineering.

[6]  Stefano Bromuri,et al.  Situating Cognitive Agents in GOLEM , 2008, EEMMAS.

[7]  D. Cook,et al.  Smart Home-Based Health Platform for Behavioral Monitoring and Alteration of Diabetes Patients , 2009, Journal of diabetes science and technology.

[8]  François Brémond,et al.  Automatic Video Interpretation: A Recognition Algorithm for Temporal Scenarios Based on Pre-compiled Scenario Models , 2003, ICVS.

[9]  Truyen Tran,et al.  Hierarchical semi-Markov conditional random fields for deep recursive sequential data , 2008, Artif. Intell..

[10]  Oren Etzioni,et al.  A Sound and Fast Goal Recognizer , 1995, IJCAI.

[11]  Fei Liu,et al.  HealthyLife: An Activity Recognition System with Smartphone Using Logic-Based Stream Reasoning , 2012, MobiQuitous.

[12]  R. Price,et al.  Artemether-lumefantrine treatment of uncomplicated Plasmodium falciparum malaria: a systematic review and meta-analysis of day 7 lumefantrine concentrations and therapeutic response using individual patient data , 2015, BMC Medicine.

[13]  Rama Chellappa,et al.  Machine Recognition of Human Activities: A Survey , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Esther Rodríguez-Villegas,et al.  COMMODITY12: A smart e-health environment for diabetes management , 2013, J. Ambient Intell. Smart Environ..

[15]  François Brémond,et al.  Automatic Video Interpretation: A Novel Algorithm for Temporal Scenario Recognition , 2003, IJCAI.

[16]  Paul J. M. Havinga,et al.  Activity Recognition Using Inertial Sensing for Healthcare, Wellbeing and Sports Applications: A Survey , 2010, ARCS Workshops.

[17]  Henry A. Kautz A formal theory of plan recognition , 1987 .

[18]  Luis Miguel Bergasa,et al.  Expert video-surveillance system for real-time detection of suspicious behaviors in shopping malls , 2015, Expert Syst. Appl..

[19]  Christopher W. Geib,et al.  Architectures for Activity Recognition and Context-Aware Computing , 2015, AI Mag..

[20]  Luca Chittaro,et al.  Modeling Medical Reasoning with the Event Calculus: An Application to the Management of Mechanical Ventilation , 1995, AIME.

[21]  Meir Kalech,et al.  Sequential Plan Recognition: (Extended Abstract) , 2016, AAMAS.

[22]  Oscar de Bruijn,et al.  Ambient Intelligence: Human–Agent Interactions in a Networked Community , 2006 .

[23]  Young-Koo Lee,et al.  A Framework for Supervising Lifestyle Diseases Using Long-Term Activity Monitoring , 2012, Sensors.

[24]  Svetha Venkatesh,et al.  Efficient duration and hierarchical modeling for human activity recognition , 2009, Artif. Intell..

[25]  Martin D. Levine,et al.  Automated Real-Time Detection of Potentially Suspicious Behavior in Public Transport Areas , 2013, IEEE Transactions on Intelligent Transportation Systems.

[26]  E. Chiauzzi,et al.  Patient-centered activity monitoring in the self-management of chronic health conditions , 2015, BMC Medicine.

[27]  Jian Lu,et al.  Mining Emerging Patterns for recognizing activities of multiple users in pervasive computing , 2009, 2009 6th Annual International Mobile and Ubiquitous Systems: Networking & Services, MobiQuitous.

[28]  Ramakant Nevatia,et al.  Video-based event recognition: activity representation and probabilistic recognition methods , 2004, Comput. Vis. Image Underst..

[29]  Arnold H. Buss Modeling with event graphs , 1996, Winter Simulation Conference.

[30]  J. Banks,et al.  Discrete-Event System Simulation , 1995 .

[31]  Ya'akov Gal,et al.  Plan Recognition for Exploratory Learning Environments Using Interleaved Temporal Search , 2015, AI Mag..

[32]  Paul J. M. Havinga,et al.  A Survey of Online Activity Recognition Using Mobile Phones , 2015, Sensors.

[33]  Krishna P. Gummadi,et al.  Towards Detecting Anomalous User Behavior in Online Social Networks , 2014, USENIX Security Symposium.

[34]  R. Sherwin,et al.  Hypoglycemia in Type 1 Diabetes , 2010, Diabetes.

[35]  V. Fonseca,et al.  Hypoglycemia, Diabetes, and Cardiovascular Events , 2010, Diabetes Care.

[36]  Kostas Stathis,et al.  Hydra: A hybrid diagnosis and monitoring architecture for diabetes , 2014, 2014 IEEE 16th International Conference on e-Health Networking, Applications and Services (Healthcom).

[37]  Gwenn Englebienne,et al.  Accurate activity recognition in a home setting , 2008, UbiComp.

[38]  Subhas Chandra Mukhopadhyay,et al.  Wearable Sensors for Human Activity Monitoring: A Review , 2015, IEEE Sensors Journal.

[39]  Jeremy V. Pitt,et al.  Connected communities from the standpoint of multi-agent systems , 1999, New Generation Computing.

[40]  Leslie Pack Kaelbling,et al.  Activity Recognition from Physiological Data using Conditional Random Fields , 2006 .

[41]  Marek J. Sergot,et al.  A logic-based calculus of events , 1989, New Generation Computing.

[42]  Paola Mello,et al.  Representing and monitoring social commitments using the event calculus , 2013, Autonomous Agents and Multi-Agent Systems.

[43]  Michael Beigl,et al.  Towards Collaborative Group Activity Recognition Using Mobile Devices , 2013, Mob. Networks Appl..

[44]  Eric Horvitz,et al.  Layered representations for learning and inferring office activity from multiple sensory channels , 2004, Comput. Vis. Image Underst..

[45]  Jake K. Aggarwal,et al.  Semantic Representation and Recognition of Continued and Recursive Human Activities , 2009, International Journal of Computer Vision.

[46]  Henry A. Kautz,et al.  Fine-grained activity recognition by aggregating abstract object usage , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[47]  Ramakant Nevatia,et al.  Hierarchical Language-based Representation of Events in Video Streams , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[48]  Paolo Mancarella,et al.  Computational Logic Foundations of KGP Agents , 2008, J. Artif. Intell. Res..

[49]  Alexander Artikis,et al.  Specifying norm-governed computational societies , 2009, TOCL.

[50]  Diane J. Cook,et al.  Detecting Anomalous Sensor Events in Smart Home Data for Enhancing the Living Experience , 2011, Artificial Intelligence and Smarter Living.

[51]  International Foundation for Autonomous Agents and MultiAgent Systems ( IFAAMAS ) , 2007 .

[52]  Miguel A. Labrador,et al.  A Survey on Human Activity Recognition using Wearable Sensors , 2013, IEEE Communications Surveys & Tutorials.

[53]  Hans Schaffers,et al.  Smart Cities and the Future Internet: Towards Cooperation Frameworks for Open Innovation , 2011, Future Internet Assembly.

[54]  Kostas Stathis,et al.  ORC: An Ontology Reasoning Component for diabetes , 2013 .

[55]  Fariba Sadri,et al.  Intention Recognition in Agents for Ambient Intelligence: Logic-Based Approaches , 2012, Agents and Ambient Intelligence.

[56]  James F. Allen,et al.  Actions and Events in Interval Temporal Logic , 1994 .

[57]  Ramakant Nevatia,et al.  VERL: An Ontology Framework for Representing and Annotating Video Events , 2005, IEEE Multim..

[58]  Rama Chellappa,et al.  Recognition of Multi-Object Events Using Attribute Grammars , 2006, 2006 International Conference on Image Processing.

[59]  J.K. Aggarwal,et al.  Human activity analysis , 2011, ACM Comput. Surv..

[60]  Thomas Kirste,et al.  Plan Synthesis for Probabilistic Activity Recognition , 2013, ICAART.

[61]  Michael Winikoff,et al.  Goals in agent systems: a unifying framework , 2008, AAMAS.

[62]  Diane J Cook,et al.  Assessing the Quality of Activities in a Smart Environment , 2009, Methods of Information in Medicine.

[63]  Angelo Montanari,et al.  EFFICIENT TEMPORAL REASONING IN THE CACHED EVENT CALCULUS , 1996, Comput. Intell..

[64]  Francesca Toni,et al.  Decision Making with a KGP Agent System , 2006, J. Decis. Syst..

[65]  Danny Weyns,et al.  Engineering Environment-Mediated Multi-Agent Systems, International Workshop, EEMMAS 2007, Dresden, Germany, October 5, 2007. Selected Revised and Invited Papers , 2008, EEMMAS.

[66]  Alexander Artikis,et al.  Run-time composite event recognition , 2012, DEBS.

[67]  Dov M. Gabbay,et al.  What Is Negation as Failure? , 2012, Logic Programs, Norms and Action.

[68]  Alfonso E. Romero,et al.  Activity Recognition for an Agent-Oriented Personal Health System , 2014, PRIMA.

[69]  Irfan A. Essa,et al.  Expectation grammars: leveraging high-level expectations for activity recognition , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..