MFED: A System for Monitoring Family Eating Dynamics

Obesity is a risk factor for many health issues, including heart disease, diabetes, osteoarthritis, and certain cancers. One of the primary behavioral causes, dietary intake, has proven particularly challenging to measure and track. Current behavioral science suggests that family eating dynamics (FED) have high potential to impact child and parent dietary intake, and ultimately the risk of obesity. Monitoring FED requires information about when and where eating events are occurring, the presence or absence of family members during eating events, and some person-level states such as stress, mood, and hunger. To date, there exists no system for real-time monitoring of FED. This paper presents MFED, the first of its kind of system for monitoring FED in the wild in real-time. Smart wearables and Bluetooth beacons are used to monitor and detect eating activities and the location of the users at home. A smartphone is used for the Ecological Momentary Assessment (EMA) of a number of behaviors, states, and situations. While the system itself is novel, we also present a novel and efficient algorithm for detecting eating events from wrist-worn accelerometer data. The algorithm improves eating gesture detection F1-score by 19% with less than 20% computation compared to the state-of-the-art methods. To date, the MFED system has been deployed in 20 homes with a total of 74 participants, and responses from 4750 EMA surveys have been collected. This paper describes the system components, reports on the eating detection results from the deployments, proposes two techniques for improving ground truth collection after the system is deployed, and provides an overview of the FED data, generated from the multi-component system, that can be used to model and more comprehensively understand insights into the monitoring of family eating dynamics.

[1]  Wolf-Joachim Fischer,et al.  Acoustical method for objective food intake monitoring using a wearable sensor system , 2011, 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[2]  Inseok Hwang,et al.  E-Gesture: a collaborative architecture for energy-efficient gesture recognition with hand-worn sensor and mobile devices , 2011, SenSys.

[3]  A. Cardello,et al.  Development and testing of a labeled magnitude scale of perceived satiety , 2005, Appetite.

[4]  David Ellis,et al.  Stress Detection Using Wearable Physiological Sensors , 2015, IWINAC.

[5]  Gerhard Tröster,et al.  Methods for Detection and Classification of Normal Swallowing from Muscle Activation and Sound , 2006, 2006 Pervasive Health Conference and Workshops.

[6]  David S. Ebert,et al.  An Overview of the Technology Assisted Dietary Assessment Project at Purdue University , 2010, 2010 IEEE International Symposium on Multimedia.

[7]  Wenyao Xu,et al.  Wearable Food Intake Monitoring Technologies: A Comprehensive Review , 2017, Comput..

[8]  S. Shiffman,et al.  Ecological momentary assessment. , 2008, Annual review of clinical psychology.

[9]  Gregory D. Abowd,et al.  Challenges and Opportunities in Automated Detection of Eating Activity , 2017, Mobile Health - Sensors, Analytic Methods, and Applications.

[10]  Paul Lukowicz,et al.  Analysis of Chewing Sounds for Dietary Monitoring , 2005, UbiComp.

[11]  Mi Zhang,et al.  BodyBeat: a mobile system for sensing non-speech body sounds , 2014, MobiSys.

[12]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[13]  Oliver Amft,et al.  Automatic Dietary Monitoring Using Wearable Accessories , 2018 .

[14]  Edward Sazonov,et al.  Automatic Ingestion Monitor: A Novel Wearable Device for Monitoring of Ingestive Behavior , 2014, IEEE Transactions on Biomedical Engineering.

[15]  K. Pasch,et al.  Examining the Relationships Between Family Meal Practices, Family Stressors, and the Weight of Youth in the Family , 2011, Annals of behavioral medicine : a publication of the Society of Behavioral Medicine.

[16]  C. Pacanowski,et al.  Contextual factors associated with eating in the absence of hunger among adults with obesity. , 2017, Eating behaviors.

[17]  Jindong Tan,et al.  DietCam: Automatic dietary assessment with mobile camera phones , 2012, Pervasive Mob. Comput..

[18]  Guanling Chen,et al.  Automatic Eating Detection using head-mount and wrist-worn accelerometers , 2015, 2015 17th International Conference on E-health Networking, Application & Services (HealthCom).

[19]  Majid Sarrafzadeh,et al.  A comparison of piezoelectric-based inertial sensing and audio-based detection of swallows , 2016 .

[20]  Wolf-Joachim Fischer,et al.  Food intake monitoring: an acoustical approach to automated food intake activity detection and classification of consumed food , 2012, Physiological measurement.

[21]  Mahesh Sooriyabandara,et al.  HealthyOffice: Mood recognition at work using smartphones and wearable sensors , 2016, 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[22]  Paul Lukowicz,et al.  Gesture spotting with body-worn inertial sensors to detect user activities , 2008, Pattern Recognit..

[23]  Gerhard Tröster,et al.  Recognition of dietary activity events using on-body sensors , 2008, Artif. Intell. Medicine.

[24]  Oleksandr Makeyev,et al.  Automatic identification of the number of food items in a meal using clustering techniques based on the monitoring of swallowing and chewing , 2012, Biomed. Signal Process. Control..

[25]  Michael R. Neuman,et al.  Automatic Detection of Swallowing Events by Acoustical Means for Applications of Monitoring of Ingestive Behavior , 2010, IEEE Transactions on Biomedical Engineering.

[26]  Yujie Dong,et al.  Detecting Periods of Eating During Free-Living by Tracking Wrist Motion , 2014, IEEE Journal of Biomedical and Health Informatics.

[27]  Adam W. Hoover,et al.  A New Method for Measuring Meal Intake in Humans via Automated Wrist Motion Tracking , 2012, Applied Psychophysiology and Biofeedback.

[28]  Edward Sazonov,et al.  Accelerometer-Based Detection of Food Intake in Free-Living Individuals , 2018, IEEE Sensors Journal.

[29]  T. Michael,et al.  Restrained eating in overweight children: does eating style run in families? , 2007, International journal of pediatric obesity : IJPO : an official journal of the International Association for the Study of Obesity.

[30]  Daily Associations of Stress and Eating in Mother–Child Dyads , 2017, Health education & behavior : the official publication of the Society for Public Health Education.

[31]  Gregory D. Abowd,et al.  Inferring Meal Eating Activities in Real World Settings from Ambient Sounds: A Feasibility Study , 2015, IUI.

[32]  Edward Sazonov,et al.  A novel approach for food intake detection using electroglottography , 2014, Physiological measurement.

[33]  P. Ciampolini,et al.  Automatic diet monitoring: a review of computer vision and wearable sensor-based methods , 2017, International journal of food sciences and nutrition.

[34]  Simon W. Ginzinger,et al.  No haste, more taste: An EMA study of the effects of stress, negative and positive emotions on eating behavior , 2018, Biological Psychology.

[35]  Michael A. Via,et al.  Obesity as a Disease , 2014, Current Obesity Reports.

[36]  John A. Stankovic,et al.  Harmony: A Hand Wash Monitoring and Reminder System using Smart Watches , 2015, EAI Endorsed Trans. Ambient Syst..

[37]  L. Lytle,et al.  Associations between perceived family meal environment and parent intake of fruit, vegetables, and fat. , 2003, Journal of nutrition education and behavior.

[38]  Min Zheng,et al.  Multimodality sensing for eating recognition , 2016, PervasiveHealth.

[39]  Wolf-Joachim Fischer,et al.  Food Intake Monitoring: Automated Chew Event Detection in Chewing Sounds , 2014, IEEE Journal of Biomedical and Health Informatics.

[40]  T. Kamarck,et al.  A global measure of perceived stress. , 1983, Journal of health and social behavior.

[41]  Gregory D. Abowd,et al.  Feasibility of identifying eating moments from first-person images leveraging human computation , 2013, SenseCam '13.

[42]  Gregory D. Abowd,et al.  A practical approach for recognizing eating moments with wrist-mounted inertial sensing , 2015, UbiComp.

[43]  John A. Stankovic,et al.  M^2G: A Monitor of Monitoring Systems with Ground Truth Validation Features for Research-Oriented Residential Applications , 2017, 2017 IEEE 14th International Conference on Mobile Ad Hoc and Sensor Systems (MASS).

[44]  Nicholas A Christakis,et al.  Social network concordance in food choice among spouses, friends, and siblings. , 2011, American journal of public health.

[45]  Edward Sazonov,et al.  Detection and characterization of food intake by wearable sensors , 2021, Wearable Sensors.

[46]  Jindong Liu,et al.  An Intelligent Food-Intake Monitoring System Using Wearable Sensors , 2012, 2012 Ninth International Conference on Wearable and Implantable Body Sensor Networks.

[47]  Y. Rosseel,et al.  Unfavourable family characteristics and their associations with childhood obesity: a cross-sectional study. , 2009, European eating disorders review : the journal of the Eating Disorders Association.

[48]  John A. Stankovic,et al.  MedRem: an interactive medication reminder and tracking system on wrist devices , 2016, 2016 IEEE Wireless Health (WH).

[49]  Hamed Haddadi,et al.  SensingKit: Evaluating the Sensor Power Consumption in iOS Devices , 2016, 2016 12th International Conference on Intelligent Environments (IE).

[50]  Koji Yatani,et al.  BodyScope: a wearable acoustic sensor for activity recognition , 2012, UbiComp.

[51]  Thomas E. Joiner,et al.  A measure of positive and negative affect for children: Scale development and preliminary validation. , 1999 .

[52]  Nabil Alshurafa,et al.  I sense overeating: Motif-based machine learning framework to detect overeating using wrist-worn sensing , 2018, Inf. Fusion.

[53]  Adam W. Hoover,et al.  Assessing the Accuracy of a Wrist Motion Tracking Method for Counting Bites Across Demographic and Food Variables , 2017, IEEE Journal of Biomedical and Health Informatics.

[54]  Raul I. Ramos-Garcia,et al.  Improving the Recognition of Eating Gestures Using Intergesture Sequential Dependencies , 2015, IEEE Journal of Biomedical and Health Informatics.

[55]  E. S. Sazonov,et al.  A Sensor System for Automatic Detection of Food Intake Through Non-Invasive Monitoring of Chewing , 2012, IEEE Sensors Journal.

[56]  Oliver Amft,et al.  Diet eyeglasses: Recognising food chewing using EMG and smart eyeglasses , 2016, 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[57]  Paul Johns,et al.  Predicting "About-to-Eat" Moments for Just-in-Time Eating Intervention , 2016, Digital Health.

[58]  Gabriele B. Papini,et al.  Ecological momentary assessment of food perceptions and eating behavior using a novel phone application in adults with or without obesity. , 2018, Eating behaviors.

[59]  Edison Thomaz,et al.  Detecting Eating Episodes by Tracking Jawbone Movements with a Non-Contact Wearable Sensor , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[60]  Xiaoli Li,et al.  Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition , 2015, IJCAI.

[61]  Adam W. Hoover,et al.  The Need for an mHealth Technology to Monitor Intake : The Bite Counter as a Case Study , 2017 .

[62]  Gregory D. Abowd,et al.  Technological approaches for addressing privacy concerns when recognizing eating behaviors with wearable cameras , 2013, UbiComp.

[63]  Konstantinos Kyritsis,et al.  Automated analysis of in meal eating behavior using a commercial wristband IMU sensor , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[64]  Gregory D. Abowd,et al.  EarBit: Using Wearable Sensors to Detect Eating Episodes in Unconstrained Environments , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[65]  Edward Sazonov,et al.  Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior. , 2008, Physiological measurement.

[66]  Nabil Alshurafa,et al.  When generalized eating detection machine learning models fail in the field , 2017, UbiComp/ISWC Adjunct.

[67]  Edward J. Delp,et al.  An image analysis system for dietary assessment and evaluation , 2010, 2010 IEEE International Conference on Image Processing.

[68]  Min Zheng,et al.  Recognizing Eating from Body-Worn Sensors , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[69]  J M Jakicic,et al.  Weight loss treatment influences untreated spouses and the home environment: evidence of a ripple effect , 2008, International Journal of Obesity.

[70]  Yi Zheng,et al.  Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks , 2014, WAIM.