Assistive intelligent environments for automatic health monitoring

As people grow older, they depend more heavily upon outside support for health assessment and medical care. The current healthcare infrastructure in America is widely considered to be inadequate to meet the needs of an increasingly older population. One solution, called aging in place, is to ensure that the elderly can live safely and independently in their own homes for as long as possible. Automatic health monitoring is a technological approach which helps people age in place by continuously providing key information to caregivers. In this thesis, we explore automatic health monitoring on several levels. First, we conduct a two-phased formative study to examine the work practices of professionals who currently perform in-home monitoring for elderly clients. With these findings in mind, we introduce the simultaneous tracking and activity recognition (STAR) problem, whose solution provides vital information for automatic in-home health monitoring. We describe and evaluate a particle filter approach that uses data from simple sensors commonly found in home security systems to provide room-level tracking and activity recognition. Next, we introduce the "context-aware recognition survey," a novel data collection method that helps users label anonymous episodes of activity for use as training examples in a supervised learner. Finally, we introduce the k-Edits Viterbi algorithm, which works within a Bayesian framework to automatically rate routine activities and detect irregular patterns of behavior. This thesis contributes to the field of automatic health monitoring through a combination of intensive background study, efficient approaches for location and activity inference, a novel unsupervised data collection technique, and a practical activity rating application.

[1]  Sunny Consolvo,et al.  Some Assembly Required: Supporting End-User Sensor Installation in Domestic Ubiquitous Computing Environments , 2004, UbiComp.

[2]  Wolfram Burgard,et al.  Learning motion patterns of persons for mobile service robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[3]  Sakuko Otake,et al.  Long-term remote behavioral monitoring of the elderly using sensors installed in domestic houses , 2002, Proceedings of the Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society] [Engineering in Medicine and Biology.

[4]  Daniel J. Barrett,et al.  An Introduction to Computerized Experience Sampling in Psychology , 2001 .

[5]  Takeo Kanade,et al.  Advances in Cooperative Multi-Sensor Video Surveillance , 1999 .

[6]  Kenji Mase,et al.  Activity and Location Recognition Using Wearable Sensors , 2002, IEEE Pervasive Comput..

[7]  Michael J. Black,et al.  Learning image statistics for Bayesian tracking , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[8]  Alex Pentland,et al.  Sensing and modeling human networks using the sociometer , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[9]  Stephan Merz,et al.  Model Checking , 2000 .

[10]  Jesse Hoey,et al.  A planning system based on Markov decision processes to guide people with dementia through activities of daily living , 2006, IEEE Transactions on Information Technology in Biomedicine.

[11]  Sunny Consolvo,et al.  The CareNet Display: Lessons Learned from an In Home Evaluation of an Ambient Display , 2004, UbiComp.

[12]  Elizabeth D. Mynatt,et al.  Digital Family Portrait Field Trial: Support for Aging in Place , 2005, CHI.

[13]  Henry A. Kautz,et al.  Inferring activities from interactions with objects , 2004, IEEE Pervasive Computing.

[14]  C. Atkeson,et al.  Toward the Automatic Assessment of Behavioral Distrubances of Dementia , 2003 .

[15]  Andy Hopper,et al.  Implementing a Sentient Computing System , 2001, Computer.

[16]  L. Burgio,et al.  Studying disruptive vocalization and contextual factors in the nursing home using computer-assisted real-time observation. , 1994, Journal of gerontology.

[17]  Elizabeth D. Mynatt,et al.  Digital family portraits: supporting peace of mind for extended family members , 2001, CHI.

[18]  Sara A. Bly,et al.  Technology for care networks of elders , 2004, IEEE Pervasive Computing.

[19]  A. Mihailidis,et al.  Context-aware assistive devices for older adults with dementia , 2002 .

[20]  Gregory D. Abowd,et al.  Charting past, present, and future research in ubiquitous computing , 2000, TCHI.

[21]  Emmanuel Munguia Tapia,et al.  Lessons learned using ubiquitous sensors for data collection in real homes , 2004, CHI EA '04.

[22]  Karen Zita Haigh,et al.  Learning Models of Human Behaviour with Sequential Patterns , 2002 .

[23]  Elizabeth D. Mynatt,et al.  Potential Intrusiveness of Aware Home Technology: Perceptions of Older Adults , 2004 .

[24]  R. Stewart,et al.  Medication compliance in the elderly. , 1989, The Medical clinics of North America.

[25]  A. Soule,et al.  Focus On Older People , 2005 .

[26]  Matthai Philipose,et al.  Maximum A Posteriori Path Estimation with Input Trace Perturbation: Algorithms and Application to Credible Rating of Human Routines , 2005, IJCAI.

[27]  John D. Lafferty,et al.  Statistical Models for Text Segmentation , 1999, Machine Learning.

[28]  Stanislav Kovacic,et al.  Trajectory Based Assessment of Coordinated Human Activity , 2003, ICVS.

[29]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[30]  A. Glascock,et al.  Behavioral Telemedicine: A New Approach to the Continuous Nonintrusive Monitoring of Activities of Daily Living , 2000 .

[31]  Shuichi Yoshino,et al.  A new in-door location detection method adopting learning algorithms , 2003, Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, 2003. (PerCom 2003)..

[32]  W. Cleghorn,et al.  The development of a computerized cueing device to help people with dementia to be more independent , 2001 .

[33]  Henry A. Kautz,et al.  Inferring High-Level Behavior from Low-Level Sensors , 2003, UbiComp.

[34]  Eric Horvitz,et al.  Layered representations for human activity recognition , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[35]  Chris Atkeson,et al.  A context-aware recognition survey for data collection using ubiquitous sensors in the home , 2005, CHI Extended Abstracts.

[36]  Martha E. Pollack,et al.  Execution monitoring with quantitative temporal Bayesian networks , 2002 .

[37]  Lars Erik Holmquist,et al.  Supporting group collaboration with interpersonal awareness devices , 1999, Personal Technologies.

[38]  Shlomo Argamon,et al.  Committee-Based Sample Selection for Probabilistic Classifiers , 1999, J. Artif. Intell. Res..

[39]  S. Katz,et al.  STUDIES OF ILLNESS IN THE AGED. THE INDEX OF ADL: A STANDARDIZED MEASURE OF BIOLOGICAL AND PSYCHOSOCIAL FUNCTION. , 1963, JAMA.

[40]  Irfan A. Essa,et al.  Exploiting human actions and object context for recognition tasks , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[41]  Aaron F. Bobick,et al.  A Framework for Recognizing Multi-Agent Action from Visual Evidence , 1999, AAAI/IAAI.

[42]  P. Tang,et al.  'Smart' homes and telecare for independent living , 2000, Journal of telemedicine and telecare.

[43]  Elizabeth D. Mynatt,et al.  Developing technology to support the functional independence of older adults , 2001 .

[44]  Fredrik Gustafsson,et al.  Monte Carlo data association for multiple target tracking , 2001 .

[45]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[46]  M C Chenier,et al.  Review and analysis of caregiver burden and nursing home placement. , 1997, Geriatric nursing.

[47]  M. Lawton,et al.  Methodological aspects of the study of streams of behavior in elders with dementing illness. , 1997, Alzheimer disease and associated disorders.

[48]  Dieter Fox,et al.  Bayesian Filtering for Location Estimation , 2003, IEEE Pervasive Comput..

[49]  Kent Larson,et al.  A living laboratory for the design and evaluation of ubiquitous computing technologies , 2005, CHI Extended Abstracts.

[50]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[51]  L. Burgio,et al.  Temporal patterns of disruptive vocalization in elderly nursing home residents , 2001, International journal of geriatric psychiatry.

[52]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[53]  Svetha Venkatesh,et al.  Activity recognition and abnormality detection with the switching hidden semi-Markov model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[54]  Ilkka Korhonen,et al.  Health monitoring in the home of the future. , 2003, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[55]  E Kassner,et al.  Determining comparable levels of functional disability. , 1998, Issue Brief (Public Policy Institute (American Association of Retired Persons)).

[56]  H. Yanco,et al.  Automation as Caregiver: A Survey of Issues and Technologies , 2003 .

[57]  Chad Burkey Environmental interfaces: HomeLab , 2000, CHI Extended Abstracts.

[58]  Gaetano Borriello,et al.  Location Systems for Ubiquitous Computing , 2001, Computer.

[59]  Bradley J. Rhodes,et al.  The wearable remembrance agent: A system for augmented memory , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[60]  Frank Dellaert,et al.  Efficient particle filter-based tracking of multiple interacting targets using an MRF-based motion model , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[61]  Elizabeth D. Mynatt,et al.  Increasing the opportunities for aging in place , 2000, CUU '00.

[62]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[63]  M. Lawton Aging and Performance of Home Tasks , 1990, Human factors.

[64]  Dadong Wan,et al.  Magic Medicine Cabinet: A Situated Portal for Consumer Healthcare , 1999, HUC.

[65]  John C. Rondoni,et al.  Context-Aware Experience Sampling for the Design and Study of Ubiquitous Technologies , 2003 .

[66]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[67]  Allison Woodruff,et al.  The mad hatter's cocktail party: a social mobile audio space supporting multiple simultaneous conversations , 2003, CHI '03.

[68]  Yaacov Ritov,et al.  Tracking Many Objects with Many Sensors , 1999, IJCAI.

[69]  Michael C. Mozer,et al.  The Neural Network House: An Environment that Adapts to its Inhabitants , 1998 .

[70]  Christopher G. Atkeson,et al.  The Narrator : A Daily Activity Summarizer Using Simple Sensors in an Instrumented Environment , 2003 .

[71]  George Casella,et al.  Implementations of the Monte Carlo EM Algorithm , 2001 .

[72]  Christopher G. Atkeson,et al.  Simultaneous Tracking and Activity Recognition (STAR) Using Many Anonymous, Binary Sensors , 2005, Pervasive.

[73]  T. Togawa,et al.  The concept of the home health monitoring , 2003, Proceedings 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (HealthCom).

[74]  J. Langford,et al.  Monte Carlo Hidden Markov Models , 1998 .

[75]  Barry Brumitt,et al.  EasyLiving: Technologies for Intelligent Environments , 2000, HUC.

[76]  William C. Mann,et al.  Enabling location-aware pervasive computing applications for the elderly , 2003, Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, 2003. (PerCom 2003)..

[77]  K. Buckwalter,et al.  Measuring problem behaviors in dementia: developing a methodological agenda. , 1997, ANS. Advances in nursing science.

[78]  Henry A. Kautz,et al.  Location-Based Activity Recognition using Relational Markov Networks , 2005, IJCAI.

[79]  Katia P. Sycara,et al.  Distributed Intelligent Agents , 1996, IEEE Expert.

[80]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[81]  Joelle Pineau,et al.  Towards robotic assistants in nursing homes: Challenges and results , 2003, Robotics Auton. Syst..

[82]  Alex Pentland,et al.  Auditory Context Awareness via Wearable Computing , 1998 .

[83]  Gregory D. Abowd,et al.  Living laboratories: the future computing environments group at the Georgia Institute of Technology , 2000, CHI Extended Abstracts.

[84]  L. Light Engineering in medicine and biology , 1970 .

[85]  Jeff A. Bilmes,et al.  A gentle tutorial of the em algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models , 1998 .

[86]  Ning Liu,et al.  Bathroom Activity Monitoring Based on Sound , 2005, Pervasive.

[87]  Ingrid Zukerman,et al.  Towards a Bayesian Model for Keyhole Plan Recognition in Large Domains , 1997 .

[88]  Patrick Pérez,et al.  The (MR)MTPF: particle filters to track multiple targets using multiple receivers , 2001 .

[89]  Stuart J. Russell,et al.  Object identification in a Bayesian context , 1997, IJCAI 1997.

[90]  Emmanuel,et al.  Activity recognition in the home setting using simple and ubiquitous sensors , 2003 .

[91]  Arthur D. Fisk,et al.  Functional Limitations to Daily Living Tasks in the Aged: A Focus Group Analysis , 1998, Hum. Factors.

[92]  Mari Ostendorf,et al.  From HMM's to segment models: a unified view of stochastic modeling for speech recognition , 1996, IEEE Trans. Speech Audio Process..

[93]  Sara J. Czaja Using technologies to aid the performance of home tasks. , 1997 .

[94]  Andrew McCallum,et al.  Confidence Estimation for Information Extraction , 2004, NAACL.

[95]  Henry A. Kautz,et al.  Guide: Towards Understanding Daily Life via Auto- Identification and Statistical Analysis , 2003 .

[96]  Arnaud Doucet,et al.  Markov chain Monte Carlo data association for target tracking , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[97]  Y. Bar-Shalom Tracking and data association , 1988 .

[98]  Donald E. Brown,et al.  Health-status monitoring through analysis of behavioral patterns , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[99]  Thomas R. Willemain,et al.  Assessing the Elderly: A Practical Guide to Measurement , 1982 .

[100]  Eamonn J. Keogh,et al.  Segmenting Time Series: A Survey and Novel Approach , 2002 .

[101]  Stephen S. Intille,et al.  A new research challenge: persuasive technology to motivate healthy aging , 2004, IEEE Transactions on Information Technology in Biomedicine.

[102]  D. Fox,et al.  Bayesian Techniques for Location Estimation , 2003 .