Macroscopic Human Behavior Interpretation Using Distributed Imager and Other Sensors

This paper presents BScope, a new system for interpreting human activity patterns using a sensor network. BScope provides a runtime, user-programmable framework that processes streams of timestamped sensor data along with prior context information to infer activities and generate appropriate notifications. The users of the system are able to describe human activities with high-level scripts that are directly mapped to hierarchical probabilistic grammars used to parse low-level sensor measurements into high-level distinguishable activities. Our approach is presented, though not limited, in the context of an assisted living application in which a small, privacy-preserving camera sensor network of five nodes is used to monitor activity in the entire house over a period of 25 days. Privacy is preserved by the fact that camera sensors only provide discrete high-level features, such as motion information in the form of image locations, and not actual images. In this deployment, our primary sensing modality is a distributed array of image sensors with wide-angle lenses that observe people's locations in the house during the course of the day. We demonstrate that our system can successfully generate summaries of everyday activities and trigger notifications at runtime by using more than 1.3 million location measurements acquired through our real home deployment.

[1]  T. Teixeira,et al.  Detecting Patterns for Assisted Living Using Sensor Networks: A Case Study , 2007, 2007 International Conference on Sensor Technologies and Applications (SENSORCOMM 2007).

[2]  Kamin Whitehouse,et al.  Semantic Streams: A Framework for Composable Semantic Interpretation of Sensor Data , 2006, EWSN.

[3]  Michael G. Thomason,et al.  Syntactic Pattern Recognition, An Introduction , 1978, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Justin Tatch Moore,et al.  Open colorings, the continuum and the second uncountable cardinal , 2002 .

[5]  King-Sun Fu,et al.  Syntactic Pattern Recognition And Applications , 1968 .

[6]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[7]  Emmanuel,et al.  Activity recognition in the home setting using simple and ubiquitous sensors , 2003 .

[8]  Yanjun Qi,et al.  Automated analysis of nursing home observations , 2004, IEEE Pervasive Computing.

[9]  King-Sun Fu,et al.  Syntactic Methods in Pattern Recognition , 1974, IEEE Transactions on Systems, Man, and Cybernetics.

[10]  Henry A. Kautz,et al.  Location-Based Activity Recognition using Relational Markov Networks , 2005, IJCAI.

[11]  P. S. Sastry,et al.  Abnormal activity detection in video sequences using learnt probability densities , 2003, TENCON 2003. Conference on Convergent Technologies for Asia-Pacific Region.

[12]  Matthai Philipose,et al.  Unsupervised Activity Recognition Using Automatically Mined Common Sense , 2005, AAAI.

[13]  Henry A. Kautz,et al.  Inferring activities from interactions with objects , 2004, IEEE Pervasive Computing.

[14]  Aaron F. Bobick,et al.  Recognition of Visual Activities and Interactions by Stochastic Parsing , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Jake K. Aggarwal,et al.  Event semantics in two-person interactions , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[17]  C. S. Wetherell,et al.  Probabilistic Languages: A Review and Some Open Questions , 1980, CSUR.

[18]  Eugenio Culurciello,et al.  A Lightweight Camera Sensor Network Operating on Symbolic Information , 2006 .

[19]  Monique Thonnat,et al.  Recurrent Bayesian Network for the Recognition of Human Behaviors from Video , 2003, ICVS.

[20]  Mani B. Srivastava,et al.  A dynamic operating system for sensor nodes , 2005, MobiSys '05.

[21]  Yiannis Aloimonos,et al.  View-Invariant Modeling and Recognition of Human Actions Using Grammars , 2006, WDV.

[22]  Irfan A. Essa,et al.  Recognizing multitasked activities from video using stochastic context-free grammar , 2002, AAAI/IAAI.

[23]  Andreas Savvides,et al.  Lightweight People Counting and Localizing in Indoor Spaces Using Camera Sensor Nodes , 2007, 2007 First ACM/IEEE International Conference on Distributed Smart Cameras.

[24]  Henry A. Kautz,et al.  Fine-grained activity recognition by aggregating abstract object usage , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[25]  Andreas Savvides,et al.  A sensory grammar for inferring behaviors in sensor networks , 2006, IPSN.