myBlackBox: Blackbox Mobile Cloud Systems for Personalized Unusual Event Detection

We demonstrate the feasibility of constructing a novel and practical real-world mobile cloud system, called myBlackBox, that efficiently fuses multimodal smartphone sensor data to identify and log unusual personal events in mobile users’ daily lives. The system incorporates a hybrid architectural design that combines unsupervised classification of audio, accelerometer and location data with supervised joint fusion classification to achieve high accuracy, customization, convenience and scalability. We show the feasibility of myBlackBox by implementing and evaluating this end-to-end system that combines Android smartphones with cloud servers, deployed for 15 users over a one-month period.

[1]  José Manuel Pastor,et al.  A Distributed Architecture for Multimodal Emotion Identification , 2015, PAAMS.

[2]  Ilias Maglogiannis,et al.  Emergency incidents detection in assisted living environments utilizing sound and visual perceptual components , 2009, PETRA '09.

[3]  Haim H. Permuter,et al.  Gaussian mixture models of texture and colour for image database retrieval , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[4]  James Williamson,et al.  Supporting Healthy Grocery Shopping via Mobile Augmented Reality , 2015, ACM Trans. Multim. Comput. Commun. Appl..

[5]  Tong Zhang,et al.  Fall Detection by Embedding an Accelerometer in Cellphone and Using KFD Algorithm , 2006 .

[6]  Beth Logan,et al.  Mel Frequency Cepstral Coefficients for Music Modeling , 2000, ISMIR.

[7]  Ehud Rivlin,et al.  Robust Real-Time Unusual Event Detection using Multiple Fixed-Location Monitors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  K. Shimizu,et al.  Location system for dementia wandering , 2000, Proceedings of the 22nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Cat. No.00CH37143).

[9]  Ameet Talwalkar,et al.  Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[10]  Johannes D. Krijnders,et al.  Multi-modal human aggression detection , 2016, Comput. Vis. Image Underst..

[11]  Akinori Ito,et al.  Detection of Abnormal Sound Using Multi-stage GMM for Surveillance Microphone , 2009, 2009 Fifth International Conference on Information Assurance and Security.

[12]  Nikos Fakotakis,et al.  On acoustic surveillance of hazardous situations , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[13]  Shivakant Mishra,et al.  Multi-modal fusion for flasher detection in a mobile video chat application , 2014, MobiQuitous.

[14]  H. S. Wolff,et al.  iRun: Horizontal and Vertical Shape of a Region-Based Graph Compression , 2022, Sensors.

[15]  Mirco Musolesi,et al.  Sensing meets mobile social networks: the design, implementation and evaluation of the CenceMe application , 2008, SenSys '08.

[16]  Nikos Fakotakis,et al.  Probabilistic Novelty Detection for Acoustic Surveillance Under Real-World Conditions , 2011, IEEE Transactions on Multimedia.

[17]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[18]  Junho Ahn,et al.  Personalized behavior pattern recognition and unusual event detection for mobile users , 2013, Mob. Inf. Syst..

[19]  Vincent Rialle,et al.  MyVigi : An Android Application to Detect Fall and Wandering , 2012 .

[20]  Frank Sposaro,et al.  iWander: An Android application for dementia patients , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[21]  Jiebo Luo,et al.  Tackling Mental Health by Integrating Unobtrusive Multimodal Sensing , 2015, AAAI.

[22]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[23]  Ahmad Rahmati,et al.  Users and Batteries: Interactions and Adaptive Energy Management in Mobile Systems , 2007, UbiComp.

[24]  Emiliano Miluzzo,et al.  CenceMe - Injecting Sensing Presence into Social Networking Applications , 2007, EuroSSC.

[25]  D. Signorini,et al.  Neural networks , 1995, The Lancet.