Seamless Tracing of Human Behavior Using Complementary Wearable and House-Embedded Sensors

This paper presents a multimodal system for seamless surveillance of elderly people in their living environment. The system uses simultaneously a wearable sensor network for each individual and premise-embedded sensors specific for each environment. The paper demonstrates the benefits of using complementary information from two types of mobility sensors: visual flow-based image analysis and an accelerometer-based wearable network. The paper provides results for indoor recognition of several elementary poses and outdoor recognition of complex movements. Instead of complete system description, particular attention was drawn to a polar histogram-based method of visual pose recognition, complementary use and synchronization of the data from wearable and premise-embedded networks and an automatic danger detection algorithm driven by two premise- and subject-related databases. The novelty of our approach also consists in feeding the databases with real-life recordings from the subject, and in using the dynamic time-warping algorithm for measurements of distance between actions represented as elementary poses in behavioral records. The main results of testing our method include: 95.5% accuracy of elementary pose recognition by the video system, 96.7% accuracy of elementary pose recognition by the accelerometer-based system, 98.9% accuracy of elementary pose recognition by the combined accelerometer and video-based system, and 80% accuracy of complex outdoor activity recognition by the accelerometer-based wearable system.

[1]  D. Lee,et al.  Computer Vision-Based Human Body Segmentation and Posture Estimation , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[2]  Siu-Yeung Cho,et al.  Human action recognition employing negative space features , 2013, J. Vis. Commun. Image Represent..

[3]  Piotr Augustyniak.  Distance Measures in Behavioral Pattern Analysis , 2011 .

[4]  David C Klonoff,et al.  Diabetes and telemedicine: is the technology sound, effective, cost-effective, and practical? , 2003, Diabetes care.

[5]  Zhen Wang,et al.  uWave: Accelerometer-based Personalized Gesture Recognition and Its Applications , 2009, PerCom.

[6]  Hossein Bobarshad,et al.  Wireless Sensor Networks for Monitoring Physiological Signals of Multiple Patients , 2011, IEEE Transactions on Biomedical Circuits and Systems.

[7]  Yung-Yaw Chen,et al.  Human Posture Recognition by Simple Rules , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[8]  Maarit Kangas,et al.  Comparison of low-complexity fall detection algorithms for body attached accelerometers. , 2008, Gait & posture.

[9]  Ramón F. Brena,et al.  Long-Term Activity Recognition from Accelerometer Data , 2013 .

[10]  Ahmad Lotfi,et al.  Smart homes for the elderly dementia sufferers: identification and prediction of abnormal behaviour , 2012, J. Ambient Intell. Humaniz. Comput..

[11]  Zeeshan Syed,et al.  Clustering and Symbolic Analysis of Cardiovascular Signals: Discovery and Visualization of Medically Relevant Patterns in Long-Term Data Using Limited Prior Knowledge , 2007, EURASIP J. Adv. Signal Process..

[12]  Hong Wei,et al.  A survey of human motion analysis using depth imagery , 2013, Pattern Recognit. Lett..

[13]  L. Tarassenko,et al.  A systematic review of telemedicine interventions to support blood glucose self‐monitoring in diabetes , 2005, Diabetic medicine : a journal of the British Diabetic Association.

[14]  Alfred Mertins,et al.  Adaptive and automated ambiance surveillance and event detection for Ambient Assisted Living , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[15]  Helly Patel,et al.  Human Tracking in Video Surveillance , 2012, ACITY.

[16]  Aleksandar Milenkovic,et al.  System architecture of a wireless body area sensor network for ubiquitous health monitoring , 2005 .

[17]  Youssef Chahir,et al.  Unified framework for human behaviour recognition: An approach using 3D Zernike moments , 2013, Neurocomputing.

[18]  Wan-Young Chung,et al.  Wireless sensor network based wearable smart shirt for ubiquitous health and activity monitoring , 2009 .

[19]  Ignacio Rojas,et al.  Ambient assisted living devices interoperability based on OSGi and the X73 standard. , 2013, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[20]  Zbigniew Mikrut,et al.  Human action recognition using simple geometric features and a finite state machine , 2013 .

[21]  A. Bourke,et al.  A threshold-based fall-detection algorithm using a bi-axial gyroscope sensor. , 2008, Medical engineering & physics.

[22]  Sridha Sridharan,et al.  An adaptive optical flow technique for person tracking systems , 2007, Pattern Recognit. Lett..

[23]  Teddy Mantoro,et al.  A Comparison Study of Classifier Algorithms for Mobile-phone's Accelerometer Based Activity Recognition , 2012 .

[24]  Alex Mihailidis,et al.  A Survey on Ambient-Assisted Living Tools for Older Adults , 2013, IEEE Journal of Biomedical and Health Informatics.

[25]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[26]  Adam Bujnowski,et al.  Monitoring of a bathing person , 2012 .

[27]  Piotr Augustyniak,et al.  Ubiquitous Cardiology: Emerging Wireless Telemedical Applications , 2009 .

[28]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[29]  Berthold K. P. Horn,et al.  "Determining optical flow": A Retrospective , 1993, Artif. Intell..

[30]  S. Levinson,et al.  Considerations in dynamic time warping algorithms for discrete word recognition , 1978 .

[31]  Toshiyo Tamura,et al.  Home geriatric physiological measurements , 2012, Physiological measurement.

[32]  Eliasz Kantoch.  Technical verification of applying wearable physiological sensors in ubiquitous health monitoring , 2013, Computing in Cardiology 2013.

[33]  Ingrid Moerman,et al.  A survey on wireless body area networks , 2011, Wirel. Networks.

[34]  Zbigniew Mikrut,et al.  Video Detection Algorithm Using an Optical Flow Calculation Method , 2012, MCSS.

[35]  M. Chan,et al.  Smart homes - current features and future perspectives. , 2009, Maturitas.

[36]  Oliver Brdiczka,et al.  Learning Situation Models in a Smart Home , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[37]  Andrés Felipe Murillo-Piedrahita,et al.  Wireless Sensor Networks for Ambient Assisted Living , 2013, Sensors.

[38]  Antoni Nowakowski,et al.  Assessment of Cardiovascular Risk in Assisted Living , 2012 .

[39]  Adam Glowacz,et al.  Diagnostics of Induction Motor Based on Analysis of Acoustic Signals with the Application of Eigenvector Method and K-Nearest Neighbor Classifier , 2012 .

[40]  Rémi Ronfard,et al.  A survey of vision-based methods for action representation, segmentation and recognition , 2011, Comput. Vis. Image Underst..

[41]  Mohan M. Trivedi,et al.  Human Pose Estimation and Activity Recognition From Multi-View Videos: Comparative Explorations of Recent Developments , 2012, IEEE Journal of Selected Topics in Signal Processing.

[42]  Baozhi Chen,et al.  Transmission of Patient Vital Signs Using Wireless Body Area Networks , 2011, Mob. Networks Appl..

[43]  Ashish Pandharipande,et al.  Presence detection using wideband audio-ultrasound sensor , 2012 .

[44]  Mauro Serpelloni,et al.  Instrumented Wearable Belt for Wireless Health Monitoring , 2010 .

[45]  Ling Shao,et al.  A survey on fall detection: Principles and approaches , 2013, Neurocomputing.

[46]  Chia-Feng Juang,et al.  Human Body Posture Classification by a Neural Fuzzy Network and Home Care System Application , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[47]  Paul Rubel,et al.  Ambient Intelligence and Pervasive Architecture Designed within the EPI-MEDICS Personal ECG Monitor , 2008, Int. J. Heal. Inf. Syst. Informatics.

[48]  Majid Sarrafzadeh,et al.  Accelerometer-based on-body sensor localization for health and medical monitoring applications , 2011, Pervasive Mob. Comput..

[49]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[50]  Willis J. Tompkins,et al.  A Real-Time QRS Detection Algorithm , 1985, IEEE Transactions on Biomedical Engineering.

[51]  Z. Mikrut,et al.  A neural network approach to recognition of the selected human motion pattern , 2011 .

[52]  Chris D. Nugent,et al.  Evidential fusion of sensor data for activity recognition in smart homes , 2009, Pervasive Mob. Comput..

[53]  Kazuya Okamoto,et al.  Reliability and validity of gait analysis by android-based smartphone. , 2012, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[54]  Lei Wang,et al.  A low-cost body inertial-sensing network for practical gait discrimination of hemiplegia patients. , 2012, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[55]  V. Ostojić,et al.  Improving asthma control through telemedicine: a study of short-message service. , 2005, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[56]  Jeffrey M. Hausdorff,et al.  Evaluation of Accelerometer-Based Fall Detection Algorithms on Real-World Falls , 2012, PloS one.

[57]  Piotr Augustyniak,et al.  Data Integration in Multimodal Home Care Surveillance and Communication System , 2010 .

[58]  Harry B. Wolfe.  Cost-benefit of laboratory computer systems , 2004, Journal of Medical Systems.

[59]  Adam Glowacz,et al.  Diagnostics of Direct Current motor with application of acoustic signals, reflection coefficients and K-Nearest Neighbor classifier , 2012 .

[60]  Richard Wootton,et al.  A Review of Telemedicine and Asthma , 2003 .

[61]  Piotr Augustyniak,et al.  Wearable Mobile Network as an Integrated Part of Assisted Living Technologies , 2012, ITIB.

[62]  Martin D. Levine,et al.  Human activity recognition in videos using a single example , 2013, Image Vis. Comput..

[63]  Cem Ersoy,et al.  Wireless sensor networks for healthcare: A survey , 2010, Comput. Networks.

[64]  Svetha Venkatesh,et al.  Recognition of emergent human behaviour in a smart home: A data mining approach , 2007, Pervasive Mob. Comput..

[65]  Piotr Augustyniak,et al.  A Graph Representation of Subject's Time-State Space , 2010 .

[66]  Ingrid Moerman,et al.  A Comprehensive Survey of Wireless Body Area Networks , 2012, Journal of Medical Systems.

[67]  Mukhtiar Memon,et al.  Ambient Assisted Living Healthcare Frameworks, Platforms, Standards, and Quality Attributes , 2014, Sensors.

[68]  Moon-Que Lee,et al.  WIRELESS BIO-RADAR SENSOR FOR HEARTBEAT AND RESPIRATION DETECTION , 2008 .

[69]  Rama Chellappa,et al.  Accuracy vs Efficiency Trade-offs in Optical Flow Algorithms , 1996, Comput. Vis. Image Underst..

[70]  Chengyou Li,et al.  Human Action Recognition Based on Template Matching , 2011 .

[71]  Manuel P. Cuéllar,et al.  Online recognition of human activities and adaptation to habit changes by means of learning automata and fuzzy temporal windows , 2013, Inf. Sci..

[72]  H. S. Wolff,et al.  iRun: Horizontal and Vertical Shape of a Region-Based Graph Compression , 2022, Sensors.

[73]  A K Bourke,et al.  Evaluation of a threshold-based tri-axial accelerometer fall detection algorithm. , 2007, Gait & posture.