Automatic Dietary Monitoring Using Wearable Accessories

This chapter provides an introduction to the field of automatic dietary monitoring (ADM) that intends to derive diet-related behaviour information from unobtrusive sensors and data analysis algorithms. A conceptual gap found in most literature reviews on the relation of physiology and dietary activities is filled. A consistent knowledge-based physiological model for dietary activities is presented. A biomedical approach is adopted to retrieve phenomenological insights of the food preparation, intake, and digestion processes. A taxonomy of dietary activities and a literature review of wearable sensing approaches and dietary dimensions across all dietary activities are also presented.

[1]  F Bellisle,et al.  Why should we study human food intake behaviour? , 2003, Nutrition, metabolism, and cardiovascular diseases : NMCD.

[2]  R. Wing,et al.  Long-term weight loss maintenance. , 2005, The American journal of clinical nutrition.

[3]  Domenico Formica,et al.  An Automated System for the Analysis of Newborns’ Oral-Motor Behavior , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[4]  Jessica R L Lieffers,et al.  Use of mobile device applications in Canadian dietetic practice. , 2014, Canadian journal of dietetic practice and research : a publication of Dietitians of Canada = Revue canadienne de la pratique et de la recherche en dietetique : une publication des Dietetistes du Canada.

[5]  Oliver Amft,et al.  Diet eyeglasses: Recognising food chewing using EMG and smart eyeglasses , 2016, 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[6]  K. Tamine,et al.  Newly developed sensor sheet for measuring tongue pressure during swallowing. , 2009, Journal of prosthodontic research.

[7]  Martin W. Donner,et al.  Normal and abnormal swallowing : imaging in diagnosis and therapy , 1991 .

[8]  Norbert Wehn,et al.  Activity recognition and nutrition monitoring in every day situations with a textile capacitive neckband , 2013, UbiComp.

[9]  Meiqin Liu,et al.  Design of real-time body weight monitor systems based on smart phones , 2014, 2014 International Conference on Mechatronics and Control (ICMC).

[10]  Mi Zhang,et al.  BodyBeat: a mobile system for sensing non-speech body sounds , 2014, MobiSys.

[11]  Ieee Staff.  2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN) , 2015 .

[12]  T. Ono,et al.  Pattern of tongue pressure on hard palate during swallowing , 2004 .

[13]  René A. de Wijk,et al.  Effects of added fluids on the perception of solid food , 2006, Physiology & Behavior.

[14]  Eli Mark Gray-Stuart,et al.  Modelling food breakdown and bolus formation during mastication : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Bioprocess Engineering at Massey University, Palmerston North, New Zealand , 2016 .

[15]  Gerhard Tröster,et al.  On-Body Sensing Solutions for Automatic Dietary Monitoring , 2009, IEEE Pervasive Computing.

[16]  Oliver Amft,et al.  Monitoring Chewing and Eating in Free-Living Using Smart Eyeglasses , 2018, IEEE Journal of Biomedical and Health Informatics.

[17]  Pablo Juliano,et al.  ENGINEERING PROPERTIES OF FOODS , 2007 .

[18]  Mingui Sun,et al.  Accuracy of food portion size estimation from digital pictures acquired by a chest-worn camera , 2013, Public Health Nutrition.

[19]  Wolf-Joachim Fischer,et al.  Food intake monitoring: an acoustical approach to automated food intake activity detection and classification of consumed food , 2012, Physiological measurement.

[20]  Martin Ekström,et al.  Wearable Weight Estimation System , 2015 .

[21]  Gregory D. Abowd,et al.  A practical approach for recognizing eating moments with wrist-mounted inertial sensing , 2015, UbiComp.

[22]  Gerhard Tröster,et al.  Methods for Detection and Classification of Normal Swallowing from Muscle Activation and Sound , 2006, 2006 Pervasive Health Conference and Workshops.

[23]  A. Woda,et al.  Effects of increased hardness on jaw movement and muscle activity during chewing of visco-elastic model foods , 2001, Experimental Brain Research.

[24]  G. Block,et al.  A review of validations of dietary assessment methods. , 1982, American journal of epidemiology.

[25]  Adam W. Hoover,et al.  Examining the utility of a bite-count-based measure of eating activity in free-living human beings. , 2014, Journal of the Academy of Nutrition and Dietetics.

[26]  Jun Rekimoto,et al.  UbiComp 2005: Ubiquitous Computing, 7th International Conference, UbiComp 2005, Tokyo, Japan, September 11-14, 2005, Proceedings , 2005, UbiComp.

[27]  Gerhard Tröster,et al.  Bite Weight Prediction From Acoustic Recognition of Chewing , 2009, IEEE Transactions on Biomedical Engineering.

[28]  Guang-Zhong Yang,et al.  A pilot study to determine whether using a lightweight, wearable micro-camera improves dietary assessment accuracy and offers information on macronutrients and eating rate , 2015, British Journal of Nutrition.

[29]  Henry A. Kautz,et al.  Fine-grained activity recognition by aggregating abstract object usage , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[30]  Majid Sarrafzadeh,et al.  Monitoring eating habits using a piezoelectric sensor-based necklace , 2015, Comput. Biol. Medicine.

[31]  Eyal Dassau,et al.  Implications of Meal Library & Meal Detection to Glycemic Control of Type 1 Diabetes Mellitus through MPC Control , 2008 .

[32]  Koji Tsukada,et al.  Sensing fork and persuasive game for improving eating behavior , 2013, UbiComp.

[33]  Robert Steele,et al.  An Overview of the State of the Art of Automated Capture of Dietary Intake Information , 2015, Critical reviews in food science and nutrition.

[34]  Takeo Odaka,et al.  Evaluation of gastrointestinal motility by computerized analysis of abdominal auscultation findings , 2006, Journal of gastroenterology and hepatology.

[35]  Richard H Sandler,et al.  Gastrointestinal sounds and migrating motor complex in fasted humans , 1999, American Journal of Gastroenterology.

[36]  Paul Lukowicz,et al.  Analysis of Chewing Sounds for Dietary Monitoring , 2005, UbiComp.

[37]  Yang Gao,et al.  Assisting Food Journaling with Automatic Eating Detection , 2016, CHI Extended Abstracts.

[38]  M. Westerterp-Plantenga,et al.  Deceleration in cumulative food intake curves, changes in body temperature and diet-induced themogenesis , 1990, Physiology & Behavior.

[39]  Mingui Sun,et al.  Saliency-aware food image segmentation for personal dietary assessment using a wearable computer , 2015, Measurement science & technology.

[40]  Maysam Ghovanloo,et al.  Tracheal activity recognition based on acoustic signals , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[41]  S. Marshall,et al.  An ethical framework for automated, wearable cameras in health behavior research. , 2013, American journal of preventive medicine.

[42]  Edward Sazonov,et al.  Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior. , 2008, Physiological measurement.

[43]  Samantha Kleinberg,et al.  Automated estimation of food type and amount consumed from body-worn audio and motion sensors , 2016, UbiComp.

[44]  Paul Lukowicz,et al.  Continuous activity recognition in the kitchen using miniaturised sensor button , 2006 .

[45]  Guanling Chen,et al.  Automatic Eating Detection using head-mount and wrist-worn accelerometers , 2015, 2015 17th International Conference on E-health Networking, Application & Services (HealthCom).

[46]  V. Vance,et al.  Use of mobile device applications in Canadian dietetic practice. , 2014, Canadian journal of dietetic practice and research : a publication of Dietitians of Canada = Revue canadienne de la pratique et de la recherche en dietetique : une publication des Dietetistes du Canada.

[47]  Edward Sazonov,et al.  A novel approach for food intake detection using electroglottography , 2014, Physiological measurement.

[48]  Edward S. Sazonov,et al.  Prediction of Bodyweight and Energy Expenditure Using Point Pressure and Foot Acceleration Measurements , 2011, The open biomedical engineering journal.

[49]  Gavin Turrell,et al.  Confidence to cook vegetables and the buying habits of Australian households. , 2009, Journal of the American Dietetic Association.

[50]  H Harry Asada,et al.  Mobile monitoring with wearable photoplethysmographic biosensors. , 2003, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[51]  Yen-Chang Chen,et al.  Sensor-embedded teeth for oral activity recognition , 2013, ISWC '13.

[52]  Pascal Makris,et al.  Origin of the Sound Components During Pharyngeal Swallowing in Normal Subjects , 2008, Dysphagia.

[53]  J. Abbink,et al.  Swallowing threshold and masticatory performance in dentate adults , 2004, Physiology & Behavior.

[54]  K. Hiiemae,et al.  MECHANISMS OF FOOD REDUCTION, TRANSPORT AND DEGLUTITION: HOW THE TEXTURE OF FOOD AFFECTS FEEDING BEHAVIOR , 2004 .

[55]  Wenyao Xu,et al.  Wearable Food Intake Monitoring Technologies: A Comprehensive Review , 2017, Comput..

[56]  Paul Lukowicz,et al.  Towards wearable sensing-based assessment of fluid intake , 2010, 2010 8th IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops).

[57]  Thad Starner,et al.  Detecting Mastication: A Wearable Approach , 2015, ICMI.

[58]  Jindong Liu,et al.  An Intelligent Food-Intake Monitoring System Using Wearable Sensors , 2012, 2012 Ninth International Conference on Wearable and Implantable Body Sensor Networks.

[59]  Oliver Amft,et al.  Bite glasses: measuring chewing using emg and bone vibration in smart eyeglasses , 2016, SEMWEB.

[60]  M. A. Rao,et al.  Engineering Properties of Foods , 2014 .

[61]  G. Tröster,et al.  Temperature Profile Estimation with Smart Textiles , 2005 .

[62]  Vigneshwaran Subbaraju,et al.  The case for smartwatch-based diet monitoring , 2015, 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[63]  Oliver Amft,et al.  Personalizing 3D-Printed Smart Eyeglasses to Augment Daily Life , 2017, Computer.

[64]  Paul Lukowicz,et al.  A generic sensor fabric for multi-modal swallowing sensing in regular upper-body shirts , 2016, SEMWEB.

[65]  Edward Sazonov,et al.  A Novel Wearable Device for Food Intake and Physical Activity Recognition , 2016, Sensors.

[66]  Maysam Ghovanloo,et al.  Unobtrusive and Wearable Systems for Automatic Dietary Monitoring , 2017, IEEE Transactions on Biomedical Engineering.

[67]  Gerhard Tröster,et al.  Recognition of dietary activity events using on-body sensors , 2008, Artif. Intell. Medicine.

[68]  Wang Yi,et al.  AutoDietary: A Wearable Acoustic Sensor System for Food Intake Recognition in Daily Life , 2016, IEEE Sensors Journal.

[69]  Ali Cinar,et al.  Meal Detection in Patients With Type 1 Diabetes: A New Module for the Multivariable Adaptive Artificial Pancreas Control System , 2016, IEEE Journal of Biomedical and Health Informatics.

[70]  Oliver Amft,et al.  Regular-look eyeglasses can monitor chewing , 2016, UbiComp Adjunct.

[71]  Oliver Amft,et al.  A wearable earpad sensor for chewing monitoring , 2010, 2010 IEEE Sensors.

[72]  P. Stumbo.  New technology in dietary assessment: a review of digital methods in improving food record accuracy , 2013, Proceedings of the Nutrition Society.

[73]  Marios Anthimopoulos,et al.  Computer Vision-Based Carbohydrate Estimation for Type 1 Patients With Diabetes Using Smartphones , 2015, Journal of diabetes science and technology.

[74]  Patrick Olivier,et al.  Slice&Dice: Recognizing Food Preparation Activities Using Embedded Accelerometers , 2009, AmI.

[75]  Qian Wang,et al.  Meal Detection and Meal Size Estimation for Type 1 Diabetes Treatment: A Variable State Dimension Approach , 2015 .

[76]  Oliver Amft,et al.  Ambient, On-Body, and Implantable Monitoring Technologies to Assess Dietary Behavior , 2011 .

[77]  Wei Wang,et al.  Your Glasses Know Your Diet: Dietary Monitoring Using Electromyography Sensors , 2017, IEEE Internet of Things Journal.

[78]  Bo Dong,et al.  Wearable sensing for liquid intake monitoring via apnea detection in breathing signals , 2014 .

[79]  Majid Sarrafzadeh,et al.  A comparison of piezoelectric-based inertial sensing and audio-based detection of swallows , 2016 .

[80]  Masaki Shuzo,et al.  WEARABLE EATING HABIT SENSING USING SOUND INFORMATION , 2009 .

[81]  Min-Chun Hu,et al.  Eat as much as you can: a kinect-based facial rehabilitation game based on mouth and tongue movements , 2014, ACM Multimedia.

[82]  Paul Lukowicz,et al.  Gesture spotting with body-worn inertial sensors to detect user activities , 2008, Pattern Recognit..

[83]  Vladimir K. Makukha,et al.  The time-response characteristics of gastrointestinal motility , 2016, 2016 13th International Scientific-Technical Conference on Actual Problems of Electronics Instrument Engineering (APEIE).

[84]  E Stellar,et al.  Chews and swallows and the microstructure of eating. , 1985, The American journal of clinical nutrition.

[85]  Gerhard Tröster,et al.  Detection of eating and drinking arm gestures using inertial body-worn sensors , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[86]  Yujie Dong,et al.  Detecting Periods of Eating During Free-Living by Tracking Wrist Motion , 2014, IEEE Journal of Biomedical and Health Informatics.

[87]  Ning Zhang,et al.  iHear Food: Eating Detection Using Commodity Bluetooth Headsets , 2016, 2016 IEEE First International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE).

[88]  Adam W. Hoover,et al.  A New Method for Measuring Meal Intake in Humans via Automated Wrist Motion Tracking , 2012, Applied Psychophysiology and Biofeedback.

[89]  H. R. Kissileff,et al.  Microstructure of eating behavior in humans , 2001, Appetite.

[90]  A E Read,et al.  Postprandial mesenteric blood flow in humans: relationship to endogenous gastrointestinal hormone secretion and energy content of food. , 1995, European journal of gastroenterology & hepatology.

[91]  J. Abbink,et al.  Skull vibration during chewing of crispy food. , 2010 .

[92]  Edward Sazonov,et al.  Detection and characterization of food intake by wearable sensors , 2021, Wearable Sensors.

[93]  Wolf-Joachim Fischer,et al.  Food Intake Monitoring: Automated Chew Event Detection in Chewing Sounds , 2014, IEEE Journal of Biomedical and Health Informatics.

[94]  Soheila Eskandari.  Bite detection and differentiation using templates of wrist motion , 2013 .

[95]  B. S. Burke.  The dietary history as a tool in research , 1947 .

[96]  Tadahiro Kuroda,et al.  Eating habits monitoring using wireless wearable in-ear microphone , 2008, 2008 3rd International Symposium on Wireless Pervasive Computing.

[97]  P. Ciampolini,et al.  Automatic diet monitoring: a review of computer vision and wearable sensor-based methods , 2017, International journal of food sciences and nutrition.

[98]  Xueliang Huo,et al.  A Magneto-Inductive Sensor Based Wireless Tongue-Computer Interface , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[99]  José Miguel Aguilera,et al.  Why food microstructure , 2005 .

[100]  E T Stewart,et al.  Physiology and radiology of the normal oral and pharyngeal phases of swallowing. , 1990, AJR. American journal of roentgenology.

[101]  Patrick Olivier,et al.  Activity Recognition and Healthier Food Preparation , 2011 .

[102]  E. Lemme,et al.  Coordination of respiration and swallowing: functional pattern and relevance of vocal folds closure. , 2010, Arquivos de gastroenterologia.

[103]  Jungmin Chung,et al.  A glasses-type wearable device for monitoring the patterns of food intake and facial activity , 2017, Scientific Reports.

[104]  Koji Yatani,et al.  BodyScope: a wearable acoustic sensor for activity recognition , 2012, UbiComp.

[105]  Gerhard Tröster,et al.  Probabilistic parsing of dietary activity events , 2007, BSN.

[106]  Nicholas Gant,et al.  Wearable cameras can reduce dietary under-reporting: doubly labelled water validation of a camera-assisted 24 h recall. , 2015, The British journal of nutrition.