BodyScope: a wearable acoustic sensor for activity recognition

Accurate activity recognition enables the development of a variety of ubiquitous computing applications, such as context-aware systems, lifelogging, and personal health systems. Wearable sensing technologies can be used to gather data for activity recognition without requiring sensors to be installed in the infrastructure. However, the user may need to wear multiple sensors for accurate recognition of a larger number of different activities. We developed a wearable acoustic sensor, called BodyScope, to record the sounds produced in the user's throat area and classify them into user activities, such as eating, drinking, speaking, laughing, and coughing. The F-measure of the Support Vector Machine classification of 12 activities using only our BodyScope sensor was 79.5%. We also conducted a small-scale in-the-wild study, and found that BodyScope was able to identify four activities (eating, drinking, speaking, and laughing) at 71.5% accuracy.

[1]  Henry A. Kautz,et al.  Inferring activities from interactions with objects , 2004, IEEE Pervasive Computing.

[2]  Björn W. Schuller,et al.  Discrimination of speech and non-linguistic vocalizations by Non-Negative Matrix Factorization , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  Paul Lukowicz,et al.  Active Capacitive Sensing: Exploring a New Wearable Sensing Modality for Activity Recognition , 2010, Pervasive.

[4]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[5]  Walterio W. Mayol-Cuevas,et al.  High level activity recognition using low resolution wearable vision , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[6]  Krzysztof Z. Gajos,et al.  Platemate: crowdsourcing nutritional analysis from food photographs , 2011, UIST.

[7]  T. Morimoto,et al.  Frequency-dependent Modulation of Rhythmic Human Jaw Movements , 1984, Journal of dental research.

[8]  Dieter Fox,et al.  Location-Based Activity Recognition , 2005, KI.

[9]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[10]  Alex Pentland,et al.  Auditory Context Awareness via Wearable Computing , 1998 .

[11]  Eric C. Larson,et al.  Accurate and privacy preserving cough sensing using a low-cost microphone , 2011, UbiComp '11.

[12]  Ning Liu,et al.  Bathroom Activity Monitoring Based on Sound , 2005, Pervasive.

[13]  Emiliano Miluzzo,et al.  Pocket, Bag, Hand, etc. - Automatically Detecting Phone Context through Discovery , 2010 .

[14]  Shahram Izadi,et al.  SenseCam: A Retrospective Memory Aid , 2006, UbiComp.

[15]  James Church,et al.  Wearable sensor badge and sensor jacket for context awareness , 1999, Digest of Papers. Third International Symposium on Wearable Computers.

[16]  Gerhard Tröster,et al.  On-Body Sensing Solutions for Automatic Dietary Monitoring , 2009, IEEE Pervasive Computing.

[17]  S Niimi,et al.  The roles of the posterior cricoarytenoid and thyropharyngeus muscles in whispered speech. , 1994, Folia phoniatrica et logopaedica : official organ of the International Association of Logopedics and Phoniatrics.

[18]  Kaare Brandt Petersen,et al.  Mel Frequency Cepstral Coefficients: An Evaluation of Robustness of MP3 Encoded Music , 2006, ISMIR.

[19]  Vesa T. Peltonen,et al.  Computational auditory scene recognition , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[20]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[21]  Guy J. Brown,et al.  Computational auditory scene analysis , 1994, Comput. Speech Lang..

[22]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[23]  Paul Lukowicz,et al.  Analysis of Chewing Sounds for Dietary Monitoring , 2005, UbiComp.

[24]  Wei Pan,et al.  SoundSense: scalable sound sensing for people-centric applications on mobile phones , 2009, MobiSys '09.

[25]  Friedrich Foerster,et al.  Detection of posture and motion by accelerometry : a validation study in ambulatory monitoring , 1999 .

[26]  Guodong Guo,et al.  Content-based audio classification and retrieval by support vector machines , 2003, IEEE Trans. Neural Networks.

[27]  Zhigang Liu,et al.  The Jigsaw continuous sensing engine for mobile phone applications , 2010, SenSys '10.

[28]  Gregory D. Abowd,et al.  Farther Than You May Think: An Empirical Investigation of the Proximity of Users to Their Mobile Phones , 2006, UbiComp.

[29]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[30]  Gerhard Tröster,et al.  Methods for Detection and Classification of Normal Swallowing from Muscle Activation and Sound , 2006, 2006 Pervasive Health Conference and Workshops.

[31]  David A. van Leeuwen,et al.  Automatic detection of laughter , 2005, INTERSPEECH.

[32]  J. Michael.  Textbook of Medical Physiology , 2005 .

[33]  Kiyohiro Shikano,et al.  Non-audible murmur recognition input interface using stethoscopic microphone attached to the skin , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..