Multi-modal Person Localization And Emergency Detection Using The Kinect

Person localization is of paramount importance in an ambient intelligence environment since it is the first step towards context-awareness. In this work, we present the development of a novel system for multi-modal person localization and emergency detection in an assistive ambient intelligence environment for the elderly. Our system is based on the depth sensor and microphone array of 2 Kinect devices. We use skeletal tracking conducted on the depth images and sound source localization conducted on the captured audio signal to estimate the location of a person. In conjunction with the location information, automatic speech recognition is used as a natural and intuitive means of communication in order to detect emergencies and accidents, such as falls. Our system attained high accuracy for both the localization and speech recognition tasks, verifying its effectiveness. Keywords-localization; multi-modal; Kinect; speech recognition; context-awareness; 3-D interaction

[1]  J. Gerberding,et al.  Fatalities and injuries from falls among older adults--United States, 1993-2003 and 2001-2005. , 2006, MMWR. Morbidity and mortality weekly report.

[2]  Bastian Leibe,et al.  Multi-person Tracking with Sparse Detection and Continuous Segmentation , 2010, ECCV.

[3]  Stephen J. McKenna,et al.  Activity summarisation and fall detection in a supportive home environment , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[4]  Ramakant Nevatia,et al.  Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet based Part Detectors , 2007, International Journal of Computer Vision.

[5]  J. Hollis,et al.  Preventing falls among community-dwelling older persons: results from a randomized trial. , 1994, The Gerontologist.

[6]  J. Krumm,et al.  Multi-camera multi-person tracking for EasyLiving , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[7]  Steve Young,et al.  The HTK book , 1995 .

[8]  Jeffrey M. Hausdorff,et al.  Gait variability and fall risk in community-living older adults: a 1-year prospective study. , 2001, Archives of physical medicine and rehabilitation.

[9]  Ramakant Nevatia,et al.  How does person identity recognition help multi-person tracking? , 2011, CVPR 2011.

[10]  Stefan Roth,et al.  People-tracking-by-detection and people-detection-by-tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Rainer Stiefelhagen,et al.  Towards vision-based 3-D people tracking in a smart room , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[12]  Luc Van Gool,et al.  Robust Multiperson Tracking from a Mobile Platform , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  C C Lau,et al.  Home accidents in elderly patients presenting to an emergency department. , 1999, Accident and emergency nursing.

[14]  K. Brown,et al.  Spacecraft hazard avoidance utilizing structured light , 2006, 2006 IEEE Aerospace Conference.

[15]  Lale Akarun,et al.  Multi-modal fall detection within the WeCare framework , 2010, IPSN '10.

[16]  Jean Meunier,et al.  Fall Detection from Human Shape and Motion History Using Video Surveillance , 2007, 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW'07).