End-to-end Learning for Measuring in-meal Eating Behavior from a Smartwatch

In this paper, we propose an end-to-end neural network (NN) architecture for detecting in-meal eating events (i.e., bites), using only a commercially available smartwatch. Our method combines convolutional and recurrent networks and is able to simultaneously learn intermediate data representations related to hand movements, as well as sequences of these movements that appear during eating. A promising F-score of 0.884 is achieved for detecting bites on a publicly available dataset with 10 subjects.

[1]  George Trigeorgis,et al.  Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Konstantinos Kyritsis,et al.  Automated analysis of in meal eating behavior using a commercial wristband IMU sensor , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[3]  Nabil Alshurafa,et al.  I sense overeating: Motif-based machine learning framework to detect overeating using wrist-worn sensing , 2018, Inf. Fusion.

[4]  N. Maglaveras,et al.  Preventing obesity and eating disorders through behavioural modifications: the SPLENDID vision , 2016, Appetite.

[5]  J. Higgins,et al.  Meal Microstructure Characterization from Sensor-Based Food Intake Detection , 2017, Front. Nutr..

[6]  Raul I. Ramos-Garcia,et al.  Improving the Recognition of Eating Gestures Using Intergesture Sequential Dependencies , 2015, IEEE Journal of Biomedical and Health Informatics.

[7]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[8]  Konstantinos Kyritsis,et al.  Food Intake Detection from Inertial Sensors Using LSTM Networks , 2017, ICIAP Workshops.

[9]  Christos Diou,et al.  A Novel Chewing Detection System Based on PPG, Audio, and Accelerometry , 2017, IEEE Journal of Biomedical and Health Informatics.

[10]  A. Astrup,et al.  Obesity : Preventing and managing the global epidemic , 2000 .

[11]  Oliver Amft,et al.  Monitoring Chewing and Eating in Free-Living Using Smart Eyeglasses , 2018, IEEE Journal of Biomedical and Health Informatics.

[12]  D A Schoeller,et al.  How accurate is self-reported dietary energy intake? , 2009, Nutrition reviews.

[13]  Christos Diou,et al.  A parametric Probabilistic Context-Free Grammar for food intake analysis based on continuous meal weight measurements , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[14]  Samantha Kleinberg,et al.  Automated estimation of food type and amount consumed from body-worn audio and motion sensors , 2016, UbiComp.