homeSound: Real-Time Audio Event Detection Based on High Performance Computing for Behaviour and Surveillance Remote Monitoring

The consistent growth in human life expectancy during the recent years has driven governments and private organizations to increase the efforts in caring for the eldest segment of the population. These institutions have built hospitals and retirement homes that have been rapidly overfilled, making their associated maintenance and operating costs prohibitive. The latest advances in technology and communications envisage new ways to monitor those people with special needs at their own home, increasing their quality of life in a cost-affordable way. The purpose of this paper is to present an Ambient Assisted Living (AAL) platform able to analyze, identify, and detect specific acoustic events happening in daily life environments, which enables the medic staff to remotely track the status of every patient in real-time. Additionally, this tele-care proposal is validated through a proof-of-concept experiment that takes benefit of the capabilities of the NVIDIA Graphical Processing Unit running on a Jetson TK1 board to locally detect acoustic events. Conducted experiments demonstrate the feasibility of this approach by reaching an overall accuracy of 82% when identifying a set of 14 indoor environment events related to the domestic surveillance and patients’ behaviour monitoring field. Obtained results encourage practitioners to keep working in this direction, and enable health care providers to remotely track the status of their patients in real-time with non-invasive methods.

[1]  Michel Vacher,et al.  On-line human activity recognition from audio and home automation sensors: Comparison of sequential and non-sequential models in realistic Smart Homes , 2016, J. Ambient Intell. Smart Environ..

[2]  Colin Mathers,et al.  The health of aging populations in China and India. , 2008, Health affairs.

[3]  Dominique Houzet,et al.  Session 4: Signal and image processing on GPU , 2011 .

[4]  Diane J. Cook,et al.  Keeping the Resident in the Loop: Adapting the Smart Home to the User , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[5]  Steven J. Miller,et al.  Using sensor networks to detect urinary tract infections in older adults , 2011, 2011 IEEE 13th International Conference on e-Health Networking, Applications and Services.

[6]  Abdenour Bouzouane,et al.  A KEYHOLE PLAN RECOGNITION MODEL FOR ALZHEIMER'S PATIENTS: FIRST RESULTS , 2007, Appl. Artif. Intell..

[7]  Georgi Gaydadjiev,et al.  A Minimalistic Architecture for Reconfigurable WFS-Based Immersive-Audio , 2010, 2010 International Conference on Reconfigurable Computing and FPGAs.

[8]  Juho Jäälinoja.  Requirements implementation in embedded software development , 2004 .

[9]  M. A. Siegler,et al.  Automatic Segmentation, Classification and Clustering of Broadcast News Audio , 1997 .

[10]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[11]  Ester Creixell Mediante,et al.  A method for recognition of coexisting environmental sound sources based on the Fisher’s linear discriminant classifier , 2015 .

[12]  Khaled Ben Letaief,et al.  Mobile Edge Computing: Survey and Research Outlook , 2017, ArXiv.

[13]  Jacques Demongeot,et al.  A model for the measurement of patient activity in a hospital suite , 2006, IEEE Transactions on Information Technology in Biomedicine.

[14]  Tero Kivimäki,et al.  Technologies for Ambient Assisted Living: Ambient Communication and Indoor Positioning , 2015 .

[15]  Sacha Krstulovic,et al.  Automatic Environmental Sound Recognition: Performance Versus Computational Cost , 2016, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[16]  Keng Peng Tee,et al.  Real-time system-level implementation of a telepresence robot using an embedded GPU platform , 2016, 2016 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[17]  Toshiyo Tamura,et al.  E-Healthcare at an Experimental Welfare Techno House in Japan , 2007, The open medical informatics journal.

[18]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[19]  P. Mermelstein,et al.  Distance measures for speech recognition, psychological and instrumental , 1976 .

[20]  Niko Moritz,et al.  Acoustic user interfaces for ambient-assisted living technologies , 2010, Informatics for health & social care.

[21]  Vesa Välimäki,et al.  GPU-Based Dynamic Wave Field Synthesis Using Fractional Delay Filters and Room Compensation , 2017, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[22]  Jie Cheng,et al.  Programming Massively Parallel Processors. A Hands-on Approach , 2010, Scalable Comput. Pract. Exp..

[23]  Sherief Reda,et al.  Hardware acceleration of feature detection and description algorithms on low-power embedded platforms , 2016, 2016 26th International Conference on Field Programmable Logic and Applications (FPL).

[24]  José Ranilla,et al.  Parallel online time warping for real-time audio-to-score alignment in multi-core systems , 2016, The Journal of Supercomputing.

[25]  Kosai Raoof,et al.  A novel acoustic indoor localization system employing CDMA , 2012, Digit. Signal Process..

[26]  Kurt Keutzer,et al.  The Concurrency Challenge , 2008, IEEE Design & Test of Computers.

[27]  Jenny Benois-Pineau,et al.  The IMMED project: wearable video monitoring of people with age dementia , 2010, ACM Multimedia.

[28]  Andrey Temko,et al.  Acoustic Event Detection and Classification , 2007, Computers in the Human Interaction Loop.

[29]  Alex Mihailidis,et al.  A Survey on Ambient-Assisted Living Tools for Older Adults , 2013, IEEE Journal of Biomedical and Health Informatics.

[30]  Andrea Lockerd Thomaz,et al.  Touched by a robot: An investigation of subjective responses to robot-initiated touch , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[31]  G. Lafortune,et al.  Trends in Severe Disability Among Elderly People: Assessing the Evidence in 12 OECD Countries and the Future Implications , 2007 .

[32]  Zdenek Becvar,et al.  Mobile Edge Computing: A Survey on Architecture and Computation Offloading , 2017, IEEE Communications Surveys & Tutorials.

[33]  Tatsuya Yamazaki,et al.  The Ubiquitous Home , 2007 .

[34]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[35]  Tuomas Virtanen,et al.  Acoustic event detection in real life recordings , 2010, 2010 18th European Signal Processing Conference.

[36]  Nathalie Japkowicz,et al.  The class imbalance problem: A systematic study , 2002, Intell. Data Anal..

[37]  Alexander H. Waibel,et al.  Temporal ICA for classification of acoustic events i a kitchen environment , 2005, INTERSPEECH.

[38]  Xavier Martorell,et al.  Work-efficient parallel non-maximum suppression for embedded GPU architectures , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[39]  Somaya Ben Allouch,et al.  An extended view on benefits and barriers of ambient assisted living solutions , 2015 .

[40]  N. M. Barnes,et al.  Lifestyle monitoring-technology for supported independence , 1998 .

[41]  Joan Claudi Socoró,et al.  A Review of Physical and Perceptual Feature Extraction Techniques for Speech, Music and Environmental Sounds , 2016 .

[42]  Sumei Liang,et al.  Audio Content Classification Method Research Based on Two-step Strategy , 2014 .

[43]  Agustín Zaballos,et al.  Solutions to the Computer Networking Challenges of the Distribution Smart Grid , 2013, IEEE Communications Letters.

[44]  Francesc Alías,et al.  Two-step detection of water sound events for the diagnostic and monitoring of dementia , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[45]  Diane J. Cook,et al.  Smart environments - technology, protocols and applications , 2004 .

[46]  Rosa Ma Alsina-Pagès,et al.  Automated Audio Data Monitoring for a Social Robot in Ambient Assisted Living Environments , 2016 .

[47]  Annamaria Mesaros,et al.  Metrics for Polyphonic Sound Event Detection , 2016 .

[48]  Andrew W. Moore,et al.  X-means: Extending K-means with Efficient Estimation of the Number of Clusters , 2000, ICML.

[49]  Daniel P. W. Ellis,et al.  Detecting Alarm Sounds , 2001 .

[50]  Georgi Gaydadjiev,et al.  Multi-Core Platforms for Beamforming and Wave Field Synthesis , 2011, IEEE Transactions on Multimedia.

[51]  Fatih Erden,et al.  Sensors in Assisted Living: A survey of signal and image processing methods , 2016, IEEE Signal Processing Magazine.

[52]  Joan Claudi Socoró,et al.  Description of Anomalous Noise Events for Reliable Dynamic Traffic Noise Mapping in Real-Life Urban and Suburban Soundscapes , 2017 .

[53]  Gustavo E. A. P. A. Batista,et al.  Class Imbalances versus Class Overlapping: An Analysis of a Learning System Behavior , 2004, MICAI.

[54]  Eric Campo,et al.  A review of smart homes - Present state and future challenges , 2008, Comput. Methods Programs Biomed..

[55]  Ilias Maglogiannis,et al.  Emergency Fall Incidents Detection in Assisted Living Environments Utilizing Motion, Sound, and Visual Perceptual Components , 2011, IEEE Transactions on Information Technology in Biomedicine.

[56]  Misha Pavel,et al.  Detection of Movement in Bed Using Unobtrusive Load Cell Sensors , 2010, IEEE Transactions on Information Technology in Biomedicine.

[57]  Joan Navarro,et al.  homeSound: A High Performance Platform for Massive Data Acquisition and Processing in Ambient Assisted Living Environments , 2017, SENSORNETS.

[58]  Gregory D. Abowd,et al.  Designing for the Human Experience in Smart Environments , 2005 .

[59]  Bart Vanrumste,et al.  Automatic Monitoring of Activities of Daily Living based on Real-life Acoustic Sensor Data: a~preliminary study , 2013, SLPAT.

[60]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[61]  Woon-Seng Gan,et al.  Fast and efficient real-time GPU based implementation of wave field synthesis , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[62]  Guiomar Corral,et al.  Security issues and threats that may affect the hybrid cloud of FINESCE , 2016, Netw. Protoc. Algorithms.

[63]  M. Popescu,et al.  Acoustic fall detection using one-class classifiers , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[64]  Yves Le Traon,et al.  Privacy challenges in Ambient Intelligence systems , 2016, J. Ambient Intell. Smart Environ..

[65]  Michel Vacher,et al.  CIRDO: Smart companion for helping elderly to live at home for longer ☆ , 2014 .

[66]  Roger Orpwood,et al.  The installation and support of internationally distributed equipment for people with dementia , 2004, IEEE Transactions on Information Technology in Biomedicine.

[67]  N. Scaringella,et al.  Automatic genre classification of music content: a survey , 2006, IEEE Signal Process. Mag..

[68]  Tom R. Halfhill.  NVIDIA's Next-Generation CUDA Compute and Graphics Architecture, Code-Named Fermi, Adds Muscle for Parallel Processing , 2009 .

[69]  James R. Larus,et al.  Software and the Concurrency Revolution , 2005, ACM Queue.

[70]  N. Grassly,et al.  United Nations Department of Economic and Social Affairs/population Division , 2022 .

[71]  Jhing-Fa Wang,et al.  Environmental Sound Classification using Hybrid SVM/KNN Classifier and MPEG-7 Audio Low-Level Descriptor , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[72]  N. Noury,et al.  Challenges in the processing of audio channels for Ambient Assisted Living , 2010, The 12th IEEE International Conference on e-Health Networking, Applications and Services.