NeckSense

We present the design, implementation, and evaluation of a multi-sensor low-power necklace 'NeckSense' for automatically and unobtrusively capturing fine-grained information about an individual's eating activity and eating episodes, across an entire waking-day in a naturalistic setting. The NeckSense fuses and classifies the proximity of the necklace from the chin, the ambient light, the Lean Forward Angle, and the energy signals to determine chewing sequences, a building block of the eating activity. It then clusters the identified chewing sequences to determine eating episodes. We tested NeckSense with 11 obese and 9 non-obese participants across two studies, where we collected more than 470 hours of data in naturalistic setting. Our result demonstrates that NeckSense enables reliable eating-detection for an entire waking-day, even in free-living environments. Overall, our system achieves an F1-score of 81.6% in detecting eating episodes in an exploratory study. Moreover, our system can achieve a F1-score of 77.1% for episodes even in an all-day-around free-living setting. With more than 15.8 hours of battery-life NeckSense will allow researchers and dietitians to better understand natural chewing and eating behaviors, and also enable real-time interventions.

[1]  Edison Thomaz,et al.  Detecting Eating Episodes by Tracking Jawbone Movements with a Non-Contact Wearable Sensor , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[2]  Nabil Alshurafa,et al.  I Can't Be Myself , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[3]  Christos Diou,et al.  A Novel Chewing Detection System Based on PPG, Audio, and Accelerometry , 2017, IEEE Journal of Biomedical and Health Informatics.

[4]  M. McCrory,et al.  Reduction of energy intake using just‐in‐time feedback from a wearable sensor system , 2017, Obesity.

[5]  Edward Sazonov,et al.  Automatic Ingestion Monitor: A Novel Wearable Device for Monitoring of Ingestive Behavior , 2014, IEEE Transactions on Biomedical Engineering.

[6]  Adam W. Hoover,et al.  A New Method for Measuring Meal Intake in Humans via Automated Wrist Motion Tracking , 2012, Applied Psychophysiology and Biofeedback.

[7]  Beat Gfeller Finding Longest Approximate Periodic Patterns , 2011, WADS.

[8]  Wendy Nilsen,et al.  Dynamic Models of Behavior for Just-in-Time Adaptive Interventions , 2014, IEEE Pervasive Computing.

[9]  L M Gallo,et al.  Time-Frequency Analysis of Chewing Activity in the Natural Environment , 2011, Journal of dental research.

[10]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[11]  H. Miura,et al.  Association of mastication and factors affecting masticatory function with obesity in adults: a systematic review , 2018, BMC Oral Health.

[12]  F. Bellisle,et al.  Impact of the daily meal pattern on energy balance 1 By France , 2004 .

[13]  Jason Tang,et al.  How can weight-loss app designers' best engage and support users? A qualitative investigation. , 2015, British journal of health psychology.

[14]  Nabil Alshurafa,et al.  Investigating barriers and facilitators to wearable adherence in fine-grained eating detection , 2017, 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops).

[15]  Mi Zhang,et al.  BodyBeat: a mobile system for sensing non-speech body sounds , 2014, MobiSys.

[16]  Min Zheng,et al.  Recognizing Eating from Body-Worn Sensors , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[17]  Nabil Alshurafa,et al.  To Mask or Not to Mask? , 2019, Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies.

[18]  Yujie Dong,et al.  Detecting Periods of Eating During Free-Living by Tracking Wrist Motion , 2014, IEEE Journal of Biomedical and Health Informatics.

[19]  B H Marcus,et al.  Physical activity interventions using mass media, print media, and information technology. , 1998, American journal of preventive medicine.

[20]  Polly Ryan,et al.  The efficacy of tailored interventions. , 2002, Journal of nursing scholarship : an official publication of Sigma Theta Tau International Honor Society of Nursing.

[21]  J. Friedman Greedy function approximation: A gradient boosting machine. , 2001 .

[22]  Vigneshwaran Subbaraju,et al.  Annapurna: Building a Real-World Smartwatch-Based Automated Food Journal , 2018, 2018 IEEE 19th International Symposium on "A World of Wireless, Mobile and Multimedia Networks" (WoWMoM).

[23]  Thad Starner,et al.  Detecting Mastication: A Wearable Approach , 2015, ICMI.

[24]  Jindong Liu,et al.  An Intelligent Food-Intake Monitoring System Using Wearable Sensors , 2012, 2012 Ninth International Conference on Wearable and Implantable Body Sensor Networks.

[25]  I. Huybrechts,et al.  Review and evaluation of innovative technologies for measuring diet in nutritional epidemiology. , 2012, International journal of epidemiology.

[26]  Oliver Amft,et al.  Monitoring Chewing and Eating in Free-Living Using Smart Eyeglasses , 2018, IEEE Journal of Biomedical and Health Informatics.

[27]  Edison Thomaz,et al.  Automatic, wearable-based, in-field eating detection approaches for public health research: a scoping review , 2020, npj Digital Medicine.

[28]  Anna Timperio,et al.  Characterizing eating patterns: a comparison of eating occasion definitions. , 2015, The American journal of clinical nutrition.

[29]  Majid Sarrafzadeh,et al.  Robust human intensity-varying activity recognition using Stochastic Approximation in wearable sensors , 2013, 2013 IEEE International Conference on Body Sensor Networks.

[30]  Gerhard Tröster,et al.  On-Body Sensing Solutions for Automatic Dietary Monitoring , 2009, IEEE Pervasive Computing.

[31]  Inbal Nahum-Shani,et al.  Building health behavior models to guide the development of just-in-time adaptive interventions: A pragmatic framework. , 2015, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[32]  Shahram Izadi,et al.  SenseCam: A Retrospective Memory Aid , 2006, UbiComp.

[33]  Gregory D. Abowd,et al.  Inferring Meal Eating Activities in Real World Settings from Ambient Sounds: A Feasibility Study , 2015, IUI.

[34]  Nabil Alshurafa,et al.  When generalized eating detection machine learning models fail in the field , 2017, UbiComp/ISWC Adjunct.

[35]  Nabil Alshurafa,et al.  WillSense: Adherence Barriers for Passive Sensing Systems That Track Eating Behavior , 2017, CHI Extended Abstracts.

[36]  Tadahiro Kuroda,et al.  Eating habits monitoring using wireless wearable in-ear microphone , 2008, 2008 3rd International Symposium on Wireless Pervasive Computing.

[37]  Majid Sarrafzadeh,et al.  Monitoring eating habits using a piezoelectric sensor-based necklace , 2015, Comput. Biol. Medicine.

[38]  Gregory D. Abowd,et al.  EarBit: Using Wearable Sensors to Detect Eating Episodes in Unconstrained Environments , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[39]  Tao Wang,et al.  Auracle: Detecting Eating Episodes with an Ear-mounted Sensor , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[40]  Kathryn M. Godfrey,et al.  Just‐in‐time adaptive interventions: A novel approach for enhancing skill utilization and acquisition in cognitive behavioral therapy for eating disorders , 2018, The International journal of eating disorders.

[41]  Koji Yatani,et al.  BodyScope: a wearable acoustic sensor for activity recognition , 2012, UbiComp.

[42]  Andrew Raij,et al.  Trends in Mobile Cyber-Physical Systems for health Just-in time interventions , 2015, SoutheastCon 2015.

[43]  Edward Sazonov,et al.  Detection of chewing from piezoelectric film sensor signals using ensemble classifiers , 2016, EMBC.

[44]  A. Knowlden,et al.  Dietary Change Interventions for Undergraduate Populations: Systematic Review and Recommendations , 2017 .

[45]  Gary L. Kreps,et al.  Rethinking Communication in the E-health Era , 2003, Journal of health psychology.

[46]  Gary L. Kreps,et al.  eHealth communication and behavior change: promise and performance , 2010 .

[47]  G. O'loughlin,et al.  Using a wearable camera to increase the accuracy of dietary analysis. , 2013, American journal of preventive medicine.

[48]  Bonnie Spring,et al.  Counting Bites With Bits: Expert Workshop Addressing Calorie and Macronutrient Intake Monitoring , 2019, Journal of medical Internet research.

[49]  Adam W. Hoover,et al.  Examining the utility of a bite-count-based measure of eating activity in free-living human beings. , 2014, Journal of the Academy of Nutrition and Dietetics.

[50]  J. Seidell,et al.  The prevention of overweight and obesity in children and adolescents: a review of interventions and programmes , 2006, Obesity reviews : an official journal of the International Association for the Study of Obesity.

[51]  Gregory D. Abowd,et al.  A practical approach for recognizing eating moments with wrist-mounted inertial sensing , 2015, UbiComp.

[52]  Majid Sarrafzadeh,et al.  A Wearable Nutrition Monitoring System , 2014, 2014 11th International Conference on Wearable and Implantable Body Sensor Networks.

[53]  Bryant T Karras,et al.  Mobile eHealth Interventions for Obesity: A Timely Opportunity to Leverage Convergence Trends , 2005, Journal of medical Internet research.

[54]  Deborah Estrin,et al.  Image browsing, processing, and clustering for participatory sensing: lessons from a DietSense prototype , 2007, EmNets '07.

[55]  Edward Sazonov,et al.  Segmentation and Characterization of Chewing Bouts by Monitoring Temporalis Muscle Using Smart Glasses With Piezoelectric Sensor , 2017, IEEE Journal of Biomedical and Health Informatics.

[56]  Gregory D. Abowd,et al.  Feasibility of identifying eating moments from first-person images leveraging human computation , 2013, SenseCam '13.

[57]  Vigneshwaran Subbaraju,et al.  The case for smartwatch-based diet monitoring , 2015, 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[58]  Ambuj Tewari,et al.  Just-in-Time Adaptive Interventions (JITAIs): An Organizing Framework for Ongoing Health Behavior Support , 2014 .

[59]  L. Lissner,et al.  Dietary underreporting by obese individuals--is it specific or non-specific? , 1995, BMJ.

[60]  Gerhard Tröster,et al.  Methods for Detection and Classification of Normal Swallowing from Muscle Activation and Sound , 2006, 2006 Pervasive Health Conference and Workshops.

[61]  Edward Sazonov,et al.  Segmentation and Characterization of Chewing Bouts by Monitoring Temporalis Muscle Using Smart Glasses with Piezoelectric Sensor. , 2016, IEEE journal of biomedical and health informatics.