Detecting Social Interaction of Elderly in a Nursing Home Environment

_______________________________________________________________________ Social interaction plays an important role in our daily lives. It is one of the most important indicators of physical or mental changes in aging patients. In this paper, we investigate the problem of detecting social interaction patterns of patients in a skilled nursing facility using audio/visual records. Our studies consist of both a “wizard of Oz” study and an experimental study of various sensors and detection models for detecting and summarizing social interactions among aging patients and caregivers. We first simulate plausible sensors using human labeling on top of audio and visual data collected from a skilled nursing facility. The most useful sensors and robust detection models are determined using the simulated sensors. We then present the implementation of some real sensors based on video and audio analysis techniques and evaluate the performance of these implementations in detecting interaction. We conclude the paper with discussions and future work.

[1]  Norman I. Badler,et al.  Temporal scene analysis: conceptual descriptions of object movements. , 1975 .

[2]  Trevor Hastie,et al.  Non‐Parametric Logistic and Proportional Odds Regression , 1987 .

[3]  R. Okafor.  Maximum likelihood estimation from incomplete data , 1987 .

[4]  N. Hooyman,et al.  Social gerontology: A multidisciplinary perspective , 1988 .

[5]  Igor Kononenko,et al.  Semi-Naive Bayesian Classifier , 1991, EWSL.

[6]  Terrence J. Sejnowski,et al.  Unsupervised Discrimination of Clustered Data via Optimization of Binary Information Gain , 1992, NIPS.

[7]  B. Rovner,et al.  The role of mental morbidity in the nursing home experience. , 1992, The Gerontologist.

[8]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[9]  P. Rabins,et al.  Management of behavior disturbance in Alzheimer disease: current knowledge and future directions. , 1992, Alzheimer disease and associated disorders.

[10]  Nicholas Emler,et al.  Gossip, reputation, and social adaptation. , 1994 .

[11]  William T. Freeman,et al.  Orientation Histograms for Hand Gesture Recognition , 1995 .

[12]  Alex Pentland,et al.  Facial expression recognition using a dynamic model and motion energy , 1995, Proceedings of IEEE International Conference on Computer Vision.

[13]  J Nelson,et al.  The influence of environmental factors in incidents of disruptive behavior. , 1995, Journal of gerontological nursing.

[14]  Bradley J. Rhodes,et al.  The wearable remembrance agent: A system for augmented memory , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[15]  Franklin J. Eppig,et al.  Mental Health of Medicare Beneficiaries: 1995 , 1997, Health care financing review.

[16]  Christoph Bregler,et al.  Learning and recognizing human dynamics in video sequences , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Thomas G. Dietterich,et al.  Pruning Adaptive Boosting , 1997, ICML.

[18]  James W. Davis,et al.  The representation and recognition of human movement using temporal templates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Alexander H. Waibel,et al.  Skin-Color Modeling and Adaptation , 1998, ACCV.

[20]  R. Hamer,et al.  Psychiatric consultation in the nursing home. A survey of six states. , 1998, The American journal of geriatric psychiatry : official journal of the American Association for Geriatric Psychiatry.

[21]  T. Jebara,et al.  Action-reaction learning : analysis and synthesis of human behaviour , 1998 .

[22]  Alex Pentland,et al.  Unsupervised clustering of ambulatory audio and video , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[23]  Gregory D. Abowd,et al.  The Aware Home: A Living Laboratory for Ubiquitous Computing Research , 1999, CoBuild.

[24]  S. Lovestone,et al.  Assessment Scales in Old Age Psychiatry , 1999, Journal of neurology, neurosurgery, and psychiatry.

[25]  Andy Hopper,et al.  The Anatomy of a Context-Aware Application , 1999, Wirel. Networks.

[26]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[27]  Michael J. Black,et al.  Parameterized Modeling and Recognition of Activities , 1999, Comput. Vis. Image Underst..

[28]  Irfan A. Essa,et al.  Exploiting human actions and object context for recognition tasks , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[29]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[30]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Aaron F. Bobick,et al.  Recognition of Visual Activities and Interactions by Stochastic Parsing , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  André Gilloire,et al.  High order statistics for robust speech/non-speech detection , 2000, 2000 10th European Signal Processing Conference.

[33]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[34]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[35]  Barry Brumitt,et al.  Ubiquitous computing and the role of geometry , 2000, IEEE Wirel. Commun..

[36]  Alex Pentland,et al.  Framing through peripheral perception , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[37]  Mubarak Shah,et al.  Monitoring human behavior from video taken in an office environment , 2001, Image Vis. Comput..

[38]  Patrick Pérez,et al.  JetStream: probabilistic contour extraction with particles , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[39]  Ramakant Nevatia,et al.  Multi-agent event recognition , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[40]  Luc Van Gool,et al.  Object Tracking with an Adaptive Color-Based Particle Filter , 2002, DAGM-Symposium.

[41]  Kenji Mase,et al.  Activity and Location Recognition Using Wearable Sensors , 2002, IEEE Pervasive Comput..

[42]  Marko Grobelnik,et al.  Feature Selection Using Linear Support Vector Machines , 2002 .

[43]  Chu-Ren Huang,et al.  Conceptual Metaphors: Ontology-based Representation and Corpora Driven Mapping Principles , 2003, ACL 2003.

[44]  Stanislav Kovacic,et al.  Trajectory Based Assessment of Coordinated Human Activity , 2003, ICVS.

[45]  Konrad Tollmar,et al.  Activity Zones for Context-Aware Computing , 2003, UbiComp.

[46]  Stan Z. Li,et al.  Real-Time Face Detection Using Boosting Learning in Hierarchical Feature Spaces , 2003 .

[47]  Christopher G. Atkeson,et al.  Predicting human interruptibility with sensors: a Wizard of Oz feasibility study , 2003, CHI '03.

[48]  Sara A. Bly,et al.  Technology for care networks of elders , 2004, IEEE Pervasive Computing.

[49]  Eric Horvitz,et al.  Layered representations for learning and inferring office activity from multiple sensory channels , 2004, Comput. Vis. Image Underst..

[50]  Kunio Fukunaga,et al.  Natural Language Description of Human Activities from Video Images Based on Concept Hierarchy of Actions , 2002, International Journal of Computer Vision.

[51]  Jodi Forlizzi,et al.  Assistive Robotics and an Ecology of Elders Living Independently in Their Homes , 2004, Hum. Comput. Interact..

[52]  Samy Bengio,et al.  Automatic analysis of multimodal group actions in meetings , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Lars Erik Holmquist,et al.  Supporting group collaboration with interpersonal awareness devices , 1999, Personal Technologies.

[54]  S. Intille,et al.  Recognizing planned, multi-person action , 2022 .