A Process for Systematic Development of Symbolic Models for Activity Recognition

Several emerging approaches to activity recognition (AR) combine symbolic representation of user actions with probabilistic elements for reasoning under uncertainty. These approaches provide promising results in terms of recognition performance, coping with the uncertainty of observations, and model size explosion when complex problems are modelled. But experience has shown that it is not always intuitive to model even seemingly simple problems. To date, there are no guidelines for developing such models. To address this problem, in this work we present a development process for building symbolic models that is based on experience acquired so far as well as on existing engineering and data analysis workflows. The proposed process is a first attempt at providing structured guidelines and practices for designing, modelling, and evaluating human behaviour in the form of symbolic models for AR. As an illustration of the process, a simple example from the office domain was developed. The process was evaluated in a comparative study of an intuitive process and the proposed process. The results showed a significant improvement over the intuitive process. Furthermore, the study participants reported greater ease of use and perceived effectiveness when following the proposed process. To evaluate the applicability of the process to more complex AR problems, it was applied to a problem from the kitchen domain. The results showed that following the proposed process yielded an average accuracy of 78%. The developed model outperformed state-of-the-art methods applied to the same dataset in previous work, and it performed comparably to a symbolic model developed by a model expert without following the proposed development process.

[1]  Kristina Yordanova,et al.  Methods for engineering symbolic human behaviour models for activity recognition , 2014, Joint Workshop of the German Research Training Groups in Computer Science.

[2]  Faicel Chamroukhi,et al.  An Unsupervised Approach for Automatic Activity Recognition Based on Hidden Markov Model Regression , 2013, IEEE Transactions on Automation Science and Engineering.

[3]  J. Banks,et al.  Discrete-Event System Simulation , 1995 .

[4]  Tony Owen,et al.  The Engineering Of Knowledge-Based Systems: Theory And Practice by Avelino J. Gonzalez and Douglas D. Dankel Prentice Hall International, Hemel Hempstead, UK, 523 pages incl. index and floppy disc (£23.50) , 1994, Robotica.

[5]  Manuela M. Veloso,et al.  Conditional random fields for activity recognition , 2007, AAMAS '07.

[6]  Thomas Kirste,et al.  Towards Tool Support for Computational Causal Behavior Models for Activity Recognition , 2012, GI-Jahrestagung.

[7]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[8]  Thomas Kirste,et al.  Context aware approach for activity recognition based on precondition-effect rules , 2012, 2012 IEEE International Conference on Pervasive Computing and Communications Workshops.

[9]  Barry W. Boehm,et al.  A spiral model of software development and enhancement , 1986, Computer.

[10]  Jesse Hoey,et al.  Automated handwashing assistance for persons with dementia using video and a partially observable Markov decision process , 2010, Comput. Vis. Image Underst..

[11]  Thomas Kirste,et al.  Plan Synthesis for Probabilistic Activity Recognition , 2013, ICAART.

[12]  Raul Silaghi,et al.  Defining Model Driven Engineering Processes , 2004 .

[13]  Abdenour Bouzouane,et al.  A Possibilistic Approach for Activity Recognition in Smart Homes for Cognitive Assistance to Alzheimer’s Patients , 2011 .

[14]  Tony Gorschek,et al.  Evaluation and Measurement of Software Process Improvement—A Systematic Literature Review , 2012, IEEE Transactions on Software Engineering.

[15]  Hui Wang,et al.  Ontology-Based Learning Framework for Activity Assistance in an Adaptive Smart Home , 2011 .

[16]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[17]  Kent Lyons,et al.  GART: The Gesture and Activity Recognition Toolkit , 2007, HCI.

[18]  Chris D. Nugent,et al.  An Ontology-Based Hybrid Approach to Activity Modeling for Smart Homes , 2014, IEEE Transactions on Human-Machine Systems.

[19]  Thomas Kirste,et al.  Towards creating assistive software by employing human behavior models , 2012, J. Ambient Intell. Smart Environ..

[20]  C. L. Philip Chen,et al.  Adaptive least squares support vector machines filter for hand tremor canceling in microsurgery , 2011, Int. J. Mach. Learn. Cybern..

[21]  John Elder,et al.  Handbook of Statistical Analysis and Data Mining Applications , 2009 .

[22]  Thomas Kirste,et al.  Marginal filtering in large state spaces , 2015, Int. J. Approx. Reason..

[23]  Osman Balci,et al.  A life cycle for modeling and simulation , 2012, Simul..

[24]  Mariano Fernández-López,et al.  Ontological Engineering , 2003, Encyclopedia of Database Systems.

[25]  Thomas Kirste,et al.  Strategies for Reducing the Complexity of Symbolic Models for Activity Recognition , 2014, AIMSA.

[26]  Jaykaran How to select appropriate statistical test , 2010 .

[27]  Bastian Hartmann,et al.  Human worker activity recognition in industrial environments , 2011 .

[28]  Martin Becker,et al.  Rule-based activity recognition framework: Challenges, technique and learning , 2009, 2009 3rd International Conference on Pervasive Computing Technologies for Healthcare.

[29]  Thomas Kirste,et al.  Evaluating the robustness of activity recognition using computational causal behavior models , 2012, UbiComp '12.

[30]  Watts S. Humphrey,et al.  Statistically Managing the Software Process , 1989 .

[31]  Jessica K. Hodgins,et al.  Guide to the Carnegie Mellon University Multimodal Activity (CMU-MMAC) Database , 2008 .

[32]  Jadwiga Indulska,et al.  31. Context Awareness , 2008 .

[33]  Chris D. Nugent,et al.  Annotating Sensor Data to Identify Activities of Daily Living , 2011, ICOST.

[34]  Jean Vanderdonckt,et al.  Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, and Challenges , 2008 .

[35]  Jesse Hoey,et al.  Sensor-Based Activity Recognition , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[36]  M. Friedman The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[37]  J. Gregory Trafton,et al.  Accommodating Human Variability in Human-Robot Teams through Theory of Mind , 2011, IJCAI.

[38]  Silvia Richter,et al.  Landmark-Based Heuristics and Search Control for Automated Planning (Extended Abstract) , 2013, IJCAI.

[39]  Martin L. King,et al.  Towards a Methodology for Building Ontologies , 1995 .

[40]  David G. Stork,et al.  Pattern Classification , 1973 .

[41]  R. Likert “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[42]  Jakob Nielsen,et al.  A mathematical model of the finding of usability problems , 1993, INTERCHI.

[43]  T. Kirste,et al.  Computational State Space Models for Activity and Intention Recognition. A Feasibility Study , 2014, PloS one.

[44]  Avelino J. Gonzalez,et al.  The engineering of knowledge-based systems: theory and practice , 1993 .

[45]  Jaykaran,et al.  How to select appropriate statistical test? , 2010 .

[46]  Thomas Kirste,et al.  A middleware for rapid prototyping smart environments: experiences in research and teaching , 2010, UbiComp '10 Adjunct.

[47]  Watts S. Humphrey Personal Software Process (PSP) , 2000 .

[48]  Deborah J. Mayhew,et al.  The usability engineering lifecycle , 1999, CHI Extended Abstracts.

[49]  Ben Kröse,et al.  A sensing and annotation system for recording datasets in multiple homes , 2009 .

[50]  I. W. Molenaar,et al.  Statistics In The Social And Behavioral Sciences , 1985 .

[51]  Kristina Yordanova Human Behaviour Modelling Approach for Intention Recognition in Ambient Assisted Living , 2011, ISAmI.

[52]  David Lorge Parnas,et al.  A rational design process: How and why to fake it , 1986, IEEE Transactions on Software Engineering.

[53]  A. Ant Ozok,et al.  SURVEY DESIGN AND IMPLEMENTATION IN HCI , 2015 .

[54]  Thomas Kirste,et al.  Strategies for Modelling Human Behaviour for Activity Recognition with Precondition-Effect Rules , 2012, KI.

[55]  Mike S. Paterson Algorithms - ESA 2000 , 2003, Lecture Notes in Computer Science.

[56]  J. Gregory Trafton,et al.  ACT-R/E , 2013, HRI 2013.

[57]  Henry A. Kautz,et al.  Inferring High-Level Behavior from Low-Level Sensors , 2003, UbiComp.

[58]  Sumi Helal,et al.  The Engineering Handbook of Smart Technology for Aging, Disability, and Independence , 2008 .

[59]  Gerhard Fischer,et al.  Context-aware systems: the 'right' information, at the 'right' time, in the 'right' place, in the 'right' way, to the 'right' person , 2012, AVI.

[60]  Hector Geffner,et al.  Probabilistic Plan Recognition Using Off-the-Shelf Classical Planners , 2010, AAAI.

[61]  Fabio Paternò,et al.  ConcurTaskTrees: A Diagrammatic Notation for Specifying Task Models , 1997, INTERACT.

[62]  Jongmyung Choi,et al.  A Context-Driven Development Methodology for Context-Aware Systems , 2011, ICHIT.

[63]  Asunción Gómez-Pérez,et al.  METHONTOLOGY: From Ontological Art Towards Ontological Engineering , 1997, AAAI 1997.

[64]  Watts S. Humphrey The Personal Software Process , 1997, Proceedings Frontiers in Education 1997 27th Annual Conference. Teaching and Learning in an Era of Change.

[65]  Steffen Staab,et al.  Knowledge Processes and Ontologies , 2001, IEEE Intell. Syst..

[66]  Simon A. Dobson,et al.  USMART , 2014, ACM Trans. Interact. Intell. Syst..

[67]  Jürgen Münch,et al.  A Systematic Methodology for Developing Discrete Event Simulation Models of Software Development Processes , 2014, ArXiv.

[68]  Kristina Yordanova Modelling Human Behaviour Using Partial Order Planning Based on Atomic Action Templates , 2011, 2011 Seventh International Conference on Intelligent Environments.

[69]  Albrecht Schmidt,et al.  Designing, Developing, and Evaluating Context-Aware Systems , 2007, 2007 International Conference on Multimedia and Ubiquitous Engineering (MUE'07).

[70]  Nolberto Munier,et al.  A Strategy for Using Multicriteria Analysis in Decision-Making , 2011 .

[71]  S.J.J. Smith,et al.  Empirical Methods for Artificial Intelligence , 1995 .

[72]  Paul Lukowicz,et al.  Performance metrics for activity recognition , 2011, TIST.

[73]  Tore Dybå,et al.  An Instrument for Measuring the Key Factors of Success in Software Process Improvement , 2000, Empirical Software Engineering.

[74]  Benjamin Naumann The Architecture Of Cognition , 2016 .

[75]  Hector Geffner,et al.  Goal Recognition over POMDPs: Inferring the Intention of a POMDP Agent , 2011, IJCAI.

[76]  Watts S. Humphrey,et al.  Managing the software process , 1989, The SEI series in software engineering.

[77]  Asunción Gómez-Pérez,et al.  Ontological Engineering: With Examples from the Areas of Knowledge Management, e-Commerce and the Semantic Web , 2004, Advanced Information and Knowledge Processing.

[78]  Daniel Jackson,et al.  Rapid specification and automated generation of prompting systems to assist people with dementia , 2011, Pervasive Mob. Comput..

[79]  David Martin,et al.  Book review: The Engineering of Knowledge-based Systems Theory and Practice by Avelino J. Gonzales and Douglas D. Dankel (Prentice Hall, 1993) , 1993, SGAR.

[80]  Martin Sachenbacher,et al.  Compiling AI Engineering Models for Probabilistic Inference , 2011, KI.

[81]  Gwenn Englebienne,et al.  Accurate activity recognition in a home setting , 2008, UbiComp.

[82]  W. W. Royce,et al.  Managing the development of large software systems: concepts and techniques , 1987, ICSE '87.

[83]  Silvia Richter,et al.  The LAMA Planner: Guiding Cost-Based Anytime Planning with Landmarks , 2010, J. Artif. Intell. Res..