Recognition of Activities of Daily Living Based on Environmental Analyses Using Audio Fingerprinting Techniques: A Systematic Review

An increase in the accuracy of identification of Activities of Daily Living (ADL) is very important for different goals of Enhanced Living Environments and for Ambient Assisted Living (AAL) tasks. This increase may be achieved through identification of the surrounding environment. Although this is usually used to identify the location, ADL recognition can be improved with the identification of the sound in that particular environment. This paper reviews audio fingerprinting techniques that can be used with the acoustic data acquired from mobile devices. A comprehensive literature search was conducted in order to identify relevant English language works aimed at the identification of the environment of ADLs using data acquired with mobile devices, published between 2002 and 2017. In total, 40 studies were analyzed and selected from 115 citations. The results highlight several audio fingerprinting techniques, including Modified discrete cosine transform (MDCT), Mel-frequency cepstrum coefficients (MFCC), Principal Component Analysis (PCA), Fast Fourier Transform (FFT), Gaussian mixture models (GMM), likelihood estimation, logarithmic moduled complex lapped transform (LMCLT), support vector machine (SVM), constant Q transform (CQT), symmetric pairwise boosting (SPB), Philips robust hash (PRH), linear discriminant analysis (LDA) and discrete cosine transform (DCT).

[1]  K. Selçuk Candan,et al.  Audio assisted group detection using smartphones , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[2]  Sukmoon Chang,et al.  An efficient audio fingerprint search algorithm for music retrieval , 2013, IEEE Transactions on Consumer Electronics.

[3]  Christian Peter,et al.  The hearing trousers pocket: activity recognition by alternative sensors , 2011, PETRA '11.

[4]  Nuno M. Garcia,et al.  Identification of Activities of Daily Living Using Sensors Available in off-the-shelf Mobile Devices: Research and Hypothesis , 2016, ISAmI.

[5]  Seungjae Lee,et al.  Audio fingerprinting based on normalized spectral subband centroids , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[6]  Taesung Park,et al.  Video bookmark based on soundtrack identification and two-stage search for interactive-television , 2007, IEEE Transactions on Consumer Electronics.

[7]  Soundararajan Srinivasan,et al.  Multisensor Fusion in Smartphones for Lifestyle Monitoring , 2010, 2010 International Conference on Body Sensor Networks.

[8]  Limin Xiao,et al.  A Two-level Audio Fingerprint Retrieval Algorithm for Advertisement Audio , 2014, MoMM.

[9]  Ashok Jhunjhunwala,et al.  Scalable and robust audio fingerprinting method tolerable to time-stretching , 2015, 2015 IEEE International Conference on Digital Signal Processing (DSP).

[10]  Xiaoqing Yu,et al.  Robust audio fingerprint extraction algorithm based on 2-D chroma , 2012, 2012 International Conference on Audio, Language and Image Processing.

[11]  John H. L. Hansen,et al.  Prof-Life-Log: Analysis and classification of activities in daily audio streams , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Xiaoqing Yu,et al.  Audio fingerprinting based on harmonic enhancement and spectral subband centroid , 2011 .

[13]  Nuno M. Garcia,et al.  From Data Acquisition to Data Fusion: A Comprehensive Review and a Roadmap for the Identification of Activities of Daily Living Using Mobile Devices , 2016, Sensors.

[14]  Igor Bisio,et al.  A Television Channel Real-Time Detector using Smartphones , 2015, IEEE Transactions on Mobile Computing.

[15]  Nuno M. Garcia A Roadmap to the Design of a Personal Digital Life Coach , 2015, ICT Innovations.

[16]  Ton Kalker,et al.  An efficient database search strategy for audio fingerprinting , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[17]  Chih-Chin Liu,et al.  An efficient audio fingerprint design for MP3 music , 2011, MoMM '11.

[18]  Ton Kalker,et al.  Speed-change resistant audio fingerprinting using auto-correlation , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[19]  Ciprian Dobre,et al.  Ambient Assisted Living and Enhanced Living Environments: Principles, Technologies and Control , 2016 .

[20]  Karthikeyan Umapathy,et al.  Audio Signal Feature Extraction and Classification Using Local Discriminant Bases , 2004, IEEE Transactions on Audio, Speech, and Language Processing.

[21]  Andreas Stolcke,et al.  Robust and Efficient Multiple Alignment of Unsynchronized Meeting Recordings , 2016, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[22]  M. Sert,et al.  A Robust and Time-Efficient Fingerprinting Model for Musical Audio , 2006, 2006 IEEE International Symposium on Consumer Electronics.

[23]  Ciprian Dobre Ambient assisted living and enhanced living environments , 2017 .

[24]  Pierre Moulin,et al.  Fingerprint information maximization for content identification , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[25]  Zafar Rafii,et al.  An audio fingerprinting system for live version identification using image processing techniques , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  Christopher Howson,et al.  Fast second screen TV synchronization combining audio fingerprint technique and generalized cross correlation , 2012, 2012 IEEE Second International Conference on Consumer Electronics - Berlin (ICCE-Berlin).

[27]  Kunio Kashino,et al.  A fast audio search method based on skipping irrelevant signals by similarity upper-bound calculation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Guang-Ho Cha,et al.  An Effective and Efficient Indexing Scheme for Audio Fingerprinting , 2011, 2011 Fifth FTRA International Conference on Multimedia and Ubiquitous Engineering.

[29]  Kyoungro Yoon,et al.  Sub-fingerprint masking for a robust audio fingerprinting system in a real-noise environment for portable consumer devices , 2010, 2010 Digest of Technical Papers International Conference on Consumer Electronics (ICCE).

[30]  Joana Raquel,et al.  Smartphone Based Human Activity Prediction , 2013 .

[31]  Wei Li,et al.  Robust online music identification using spectral entropy in the compressed domain , 2014, 2014 IEEE Wireless Communications and Networking Conference Workshops (WCNCW).

[32]  N. Garcia,et al.  Multi-sensor data fusion techniques for the identification of activities of daily living using mobile devices , 2015 .

[33]  Takashi Shibuya,et al.  Audio fingerprinting robust against reverberation and noise based on quantification of sinusoidality , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[34]  Antonio Garzon,et al.  MASK: Robust Local Features for Audio Fingerprinting , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[35]  Igor Bisio,et al.  Opportunistic estimation of television audience through smartphones , 2012, 2012 International Symposium on Performance Evaluation of Computer & Telecommunication Systems (SPECTS).

[36]  Chih-Chin Liu MP3 sniffer: a system for online detecting MP3 music transmissions , 2012, MoMM '12.

[37]  Lahouari Ghouti,et al.  A Fingerprinting System for Musical Content , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[38]  Jin Young Kim,et al.  TV Advertisement Search Based on Audio Peak-Pair Hashing in Real Environments , 2015, 2015 5th International Conference on IT Convergence and Security (ICITCS).

[39]  Ricardo Costa,et al.  Ambient Assisted Living , 2009 .

[40]  Miwako Doi,et al.  Indoor-outdoor activity recognition by a smartphone , 2012, UbiComp.

[41]  Winifred Schultz-Krohn PhD Otr,et al.  Pedretti's Occupational Therapy: Practice Skills for Physical Dysfunction , 2012 .

[42]  Stephan Sigg,et al.  Secure Communication Based on Ambient Audio , 2013, IEEE Transactions on Mobile Computing.

[43]  Wei Xiong,et al.  Audio fingerprinting based on dynamic subband locating and normalized SSC , 2012, 2012 International Conference on Audio, Language and Image Processing.

[44]  Jyh-Shing Roger Jang,et al.  Speeding up audio fingerprinting over GPUs , 2014, 2014 International Conference on Audio, Language and Image Processing.

[45]  Kaichun Chang,et al.  Sub-nyquist audio fingerprinting for music recognition , 2010, 2010 2nd Computer Science and Electronic Engineering Conference (CEEC).

[46]  M. Cremer,et al.  A Tunable, Efficient, Specialized Multidimensional Range Query Algorithm , 2006, 2006 IEEE International Symposium on Signal Processing and Information Technology.

[47]  Jin S. Seo An Asymmetric Matching Method for a Robust Binary Audio Fingerprinting , 2014, IEEE Signal Processing Letters.

[48]  Cesar Pedraza,et al.  Fast parallel audio fingerprinting implementation in reconfigurable hardware and GPUs , 2011, 2011 VII Southern Conference on Programmable Logic (SPL).

[49]  Jun-Yong Lee,et al.  Audio fingerprinting to identify TV commercial advertisement in real-noisy environment , 2014, 2014 14th International Symposium on Communications and Information Technologies (ISCIT).

[50]  Xiaoqing Yu,et al.  Audio fingerprinting based on local energy centroid , 2011 .

[51]  Guang Yang,et al.  Efficient music identification by utilizing space-saving audio fingerprinting system , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).