What is happening now? Detection of activities of daily living from simple visual features

We propose and investigate a paradigm for activity recognition, distinguishing the “on-going activity” recognition task (OGA) from that addressing “complete activities” (CA). The former starts from a time interval and aims to discover which activities are going on inside it. The latter, in turn, focuses on terminated activities and amounts to taking an external perspective on activities. We argue that this distinction is quite natural and the OGA task has a number of interesting properties; e.g., the possibility of reconstructing complete activities in terms of on-going ones, the avoidance of the thorny issue of activity segmentation, and a straightforward accommodation of complex activities, etc. Moreover, some plausible properties of the OGA task are discussed and then investigated in a classification study, addressing: the dependence of classification performance on the duration of time windows and its relationship with actional types (homogeneous vs. non-homogeneous activities), and on the assortments of features used. Three types of visual features are exploited, obtained from a data set that tries to balance the pros and cons of laboratory-based and naturalistic ones. The results provide partial confirmation to the hypothesis and point to relevant open issues for future work.

[1]  Chris D. Nugent,et al.  The use of computer vision techniques to augment home based sensorised environments , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[2]  Ulrich H.-G. Kreßel,et al.  Pairwise classification and support vector machines , 1999 .

[3]  Roberto Brunelli,et al.  An Appearance-Based Particle Filter for Visual Tracking in Smart Rooms , 2007, CLEAR.

[4]  Kent Larson,et al.  Using a Live-In Laboratory for Ubiquitous Computing Research , 2006, Pervasive.

[5]  Jennifer Healey,et al.  A Long-Term Evaluation of Sensing Modalities for Activity Recognition , 2007, UbiComp.

[6]  Paul Chippendale,et al.  Towards Automatic Body Language Annotation , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[7]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[8]  S. Katz,et al.  STUDIES OF ILLNESS IN THE AGED. THE INDEX OF ADL: A STANDARDIZED MEASURE OF BIOLOGICAL AND PSYCHOSOCIAL FUNCTION. , 1963, JAMA.

[9]  Chih-Yang Lin,et al.  Finding the mitral annular lines from 2-D + 1-D precordial echocardiogram using graph-search technique , 2004, IEEE Transactions on Information Technology in Biomedicine.

[10]  Matthai Philipose,et al.  Building Reliable Activity Models Using Hierarchical Shrinkage and Mined Ontology , 2006, Pervasive.

[11]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[12]  Matthew S. Goodwin,et al.  Recognizing stereotypical motor movements in the laboratory and classroom: a case study with children on the autism spectrum , 2009, UbiComp.

[13]  Ben Taskar,et al.  Introduction to statistical relational learning , 2007 .

[14]  Irfan A. Essa,et al.  Discovering Characteristic Actions from On-Body Sensor Data , 2006, 2006 10th IEEE International Symposium on Wearable Computers.

[15]  Nello Cristianini,et al.  Advances in Kernel Methods - Support Vector Learning , 1999 .

[16]  Irfan A. Essa,et al.  A novel sequence representation for unsupervised analysis of human activities , 2009, Artif. Intell..

[17]  James M. Rehg,et al.  A Scalable Approach to Activity Recognition based on Object Use , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[18]  Gaetano Borriello,et al.  A Practical Approach to Recognizing Physical Activities , 2006, Pervasive.

[19]  Eric Horvitz,et al.  Layered representations for human activity recognition , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[20]  Fabio Pianesi,et al.  Events and Event Talk: An Introduction , 2000 .

[21]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[22]  Fabio Pianesi,et al.  Speaking of Events , 2000 .

[23]  Kent Larson,et al.  Real-Time Recognition of Physical Activities and Their Intensities Using Wireless Accelerometers and a Heart Rate Monitor , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[24]  Henry Kautz,et al.  Hierarchical recognition of activities of daily living using multi-scale, multi-perspective vision and RFID , 2008 .

[25]  Oliver Brdiczka,et al.  Detecting Human Behavior Models From Multimodal Observation in a Smart Home , 2009, IEEE Transactions on Automation Science and Engineering.

[26]  Alan Agresti,et al.  Categorical Data Analysis , 1991, International Encyclopedia of Statistical Science.

[27]  Daniel Gatica-Perez,et al.  Learning and predicting multimodal daily life patterns from cell phones , 2009, ICMI-MLMI '09.

[28]  Henry A. Kautz,et al.  Inferring activities from interactions with objects , 2004, IEEE Pervasive Computing.

[29]  Samy Bengio,et al.  Automatic analysis of multimodal group actions in meetings , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[31]  Christian Wojek,et al.  Activity Recognition and Room-Level Tracking in an Office Environment , 2006, 2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[32]  S. Katz.  Studies of illness in the aged , 1963 .

[33]  Kent Larson,et al.  The Design of a Portable Kit of Wireless Sensors for Naturalistic Data Collection , 2006, Pervasive.

[34]  Uwe Hansmann,et al.  Pervasive Computing , 2003 .

[35]  Sydney Katz.  Assessing Self‐maintenance: Activities of Daily Living, Mobility, and Instrumental Activities of Daily Living , 1983, Journal of the American Geriatrics Society.

[36]  Alex Mihailidis,et al.  The use of computer vision in an intelligent environment to support aging-in-place, safety, and independence in the home , 2004, IEEE Transactions on Information Technology in Biomedicine.

[37]  M. Ogawa,et al.  An attempt of monitoring daily activities at home , 2000, Proceedings of the 22nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Cat. No.00CH37143).

[39]  Ben Taskar,et al.  Introduction to Statistical Relational Learning (Adaptive Computation and Machine Learning) , 2007 .

[40]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[41]  Miguel Figueroa,et al.  Competitive learning with floating-gate circuits , 2002, IEEE Trans. Neural Networks.

[42]  Bernt Schiele,et al.  ADL recognition based on the combination of RFID and accelerometer sensing , 2008, 2008 Second International Conference on Pervasive Computing Technologies for Healthcare.

[43]  Jeff A. Bilmes,et al.  Learning Large Scale Common Sense Models of Everyday Life , 2007, AAAI.