Audio IoT Analytics for Home Automation Safety

The aim of the paper is to perform audio analytics based on the audio sensor data that is continuously monitoring the home environment automatically through an audio Internet of Things (IoT) system. Domestic violence is one of the major problems in many cities nowadays. We have proposed a home automation system where IoT sensors records the audio in home environment continuously and the audio is sent to machine learning server where the audio is split into small clips and classified into different categories. The need of an automatic detection system is urgent for enforcing home safety and safe neighborhood. If IoT system detects any suspicious sound, it generates an emergency notification to nearest emergency services for possible action to be taken. The classification of audio such as gunshots, explosion, glass breaking, screaming and siren is based on shallow learning (Support vector machine, Decision tree, Random forest and Naïve Bayes) and deep learning (Convolutional neural network and Long short-term memory). Our experiments validated that Convolutional Neural Network shows the best performance (89% accuracy) compared to other machine learning algorithms.

[1]  Nikhil Ketkar,et al.  Introduction to Keras , 2017 .

[2]  Takao Kobayashi,et al.  Harmonics tracking and pitch extraction based on instantaneous frequency , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[3]  Beth Logan,et al.  Mel Frequency Cepstral Coefficients for Music Modeling , 2000, ISMIR.

[4]  Karol J. Piczak Environmental sound classification with convolutional neural networks , 2015, 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP).

[5]  J. Li,et al.  Smart city and the applications , 2011, 2011 International Conference on Electronics, Communications and Control (ICECC).

[6]  Mohan S. Kankanhalli,et al.  Audio Based Event Detection for Multimedia Surveillance , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[7]  Nasser Kehtarnavaz,et al.  Smartphone-based real-time classification of noise signals using subband features and random forest classifier , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Colin Raffel,et al.  librosa: Audio and Music Signal Analysis in Python , 2015, SciPy.

[9]  Kin Hong Wong,et al.  Music genre classification using a hierarchical long short term memory (LSTM) model , 2018, International Workshop on Pattern Recognition.

[10]  Ian F. Akyildiz,et al.  Wireless Sensor Networks: Akyildiz/Wireless Sensor Networks , 2010 .

[11]  David Ellis,et al.  Stress Detection Using Wearable Physiological Sensors , 2015, IWINAC.

[12]  Heikki Huttunen,et al.  Recurrent neural networks for polyphonic sound event detection in real life recordings , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Andrea Zanella,et al.  Internet of Things for Smart Cities , 2014, IEEE Internet of Things Journal.

[14]  Lei Wang,et al.  Convolutional Recurrent Neural Networks for Text Classification , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[15]  Nicholas D. Lane,et al.  DeepEar: robust smartphone audio sensing in unconstrained acoustic environments using deep learning , 2015, UbiComp.

[16]  Yann LeCun,et al.  Convolutional neural networks applied to house numbers digit classification , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[17]  Justin Salamon,et al.  Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification , 2016, IEEE Signal Processing Letters.

[18]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[19]  Yann LeCun,et al.  Moving Beyond Feature Design: Deep Architectures and Automatic Feature Learning in Music Informatics , 2012, ISMIR.

[20]  Theresa A. Pardo,et al.  Conceptualizing smart city with dimensions of technology, people, and institutions , 2011, dg.o '11.

[21]  Can Wang,et al.  Violence detection using Oriented VIolent Flows , 2016, Image Vis. Comput..