Tracking wrist motion to detect and measure the eating intake of free-living humans

This dissertation is motivated by the growing prevalence of obesity, a health problem currently affecting over 500 million people worldwide. It is composed of two studies. In the first study, a new method is developed to detect how many bites a person takes during a meal in real time. A pattern has been found that the wrist of a person undergoes a characteristic roll motion as food is picked up and placed into the mouth. This motion can be tracked by a gyroscope sensor placed on the wrist. This work could be used in many weight loss and obesity treatment applications, including monitoring intake, slowing eating rate, and providing a cue for mindful eating. In the second study, a new method is developed to automatically distinguish eating activity from other activities in natural daily living. Accelerometers are used to detect the typical burst activity at the beginning and the end of each eating activity and gyroscope roll motion features are used during hypothesized detections to differentiate eating activities from other activities. This work has many potential applications. It could be used by individuals for self-monitoring for weight loss and weight maintenance. It could be combined with a food diary, 24-hour recall or food frequency questionnaire to improve compliance and accuracy in measuring consumption. The two methods could potentially be combined to automatically count bites of intake all day. The methods could also be used by clinical practitioners to monitor the eating patterns of patients (for example during diabetes treatment), or by researchers in epidemiological and genetic studies (for example in studies of the physical activity or eating habits of specific demographics).

[1]  Donald A Williamson,et al.  Comparison of digital photography to weighed and visual estimation of portion sizes. , 2003, Journal of the American Dietetic Association.

[2]  F. Takeda,et al.  Dish extraction method with neural network for food intake measuring system on medical use , 2003, The 3rd International Workshop on Scientific Use of Submarine Cables and Related Technologies, 2003..

[3]  Jay Stewart,et al.  The American Time Use Survey , 2005 .

[4]  Miwako Doi,et al.  LifeMinder: a wearable healthcare support system with timely instruction based on the user's context , 2004, The 8th IEEE International Workshop on Advanced Motion Control, 2004. AMC '04..

[5]  K. Westerterp,et al.  Physical Activity Assessment With Accelerometers: An Evaluation Against Doubly Labeled Water , 2007, Obesity.

[6]  Urbashi Mitra,et al.  Multimodal Physical Activity Recognition by Fusing Temporal and Cepstral Information , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  Robin Thomas,et al.  Challenges and results of sampling Chinese restaurant menu items for the USDA National Nutrient Database for Standard Reference , 2011 .

[8]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[9]  Rachel K. Johnson,et al.  The use of a personal digital assistant for dietary self-monitoring does not improve the validity of self-reports of energy intake. , 2006, Journal of the American Dietetic Association.

[10]  Bruce Howard,et al.  Lightglove: wrist-worn virtual typing and pointing , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[11]  Kamiar Aminian,et al.  Quantification of Tremor and Bradykinesia in Parkinson's Disease Using a Novel Ambulatory Monitoring System , 2007, IEEE Transactions on Biomedical Engineering.

[12]  E. Delp,et al.  Evidence-based development of a mobile telephone food record. , 2010, Journal of the American Dietetic Association.

[13]  M. Marmot,et al.  Dietary assessment in Whitehall II: comparison of 7 d diet diary and food-frequency questionnaire and validity against biomarkers , 2001, British Journal of Nutrition.

[14]  Gerhard Tröster,et al.  Recognition of dietary activity events using on-body sensors , 2008, Artif. Intell. Medicine.

[15]  McGinnis Jm,et al.  Actual causes of death in the United States. , 1993 .

[16]  D A Schoeller,et al.  Measurement of energy expenditure in free-living humans by using doubly labeled water. , 1988, The Journal of nutrition.

[17]  H. Kissileff,et al.  Universal eating monitor for continuous recording of solid or liquid consumption in man. , 1980, The American journal of physiology.

[18]  Tom Chau,et al.  Segmentation of Dual-Axis Swallowing Accelerometry Signals in Healthy Subjects With Analysis of Anthropometric Effects on Duration of Swallowing Activities , 2009, IEEE Transactions on Biomedical Engineering.

[19]  Nigel H. Lovell,et al.  Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring , 2006, IEEE Transactions on Information Technology in Biomedicine.

[20]  A F Subar,et al.  Improving food frequency questionnaires: a qualitative approach using cognitive interviewing. , 1995, Journal of the American Dietetic Association.

[21]  S. Heymsfield,et al.  Discrepancy between self-reported and actual caloric intake and exercise in obese subjects. , 1992, The New England journal of medicine.

[22]  K. Flegal,et al.  Prevalence and trends in obesity among US adults, 1999-2008. , 2010, JAMA.

[23]  Anthony Rowe,et al.  eWatch: context sensitive system design case study , 2005, IEEE Computer Society Annual Symposium on VLSI: New Frontiers in VLSI Design (ISVLSI'05).

[24]  Russell Beale,et al.  A technical review of mobile computational devices , 2003, J. Comput. Assist. Learn..

[25]  P. Kris-Etherton,et al.  Accuracy of energy intake data estimated by a multiple-pass, 24-hour dietary recall technique. , 2000, Journal of the American Dietetic Association.

[26]  Abby G. Ershow,et al.  Engineering Approaches to Energy Balance and Obesity: Opportunities for Novel Collaborations and Research , 2007, Journal of diabetes science and technology.

[27]  Roger B. Dannenberg,et al.  Learning an Orchestra Conductor's Technique Using a Wearable Sensor Platform , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[28]  C J Harland,et al.  High resolution ambulatory electrocardiographic monitoring using wrist mounted electric potential sensors , 2003 .

[29]  Gerhard Tröster,et al.  Detection of eating and drinking arm gestures using inertial body-worn sensors , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[30]  Anthony Rowe,et al.  eWatch: a wearable sensor and notification platform , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[31]  Robert R. Harris,et al.  Doubly-Labelled Water – Theory and Practice , 2001 .

[32]  R. Troiano,et al.  Psychosocial predictors of energy underreporting in a large doubly labeled water study. , 2004, The American journal of clinical nutrition.

[33]  Paul Lukowicz,et al.  Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  L. Arab,et al.  Eight self-administered 24-hour dietary recalls using the Internet are feasible in African Americans and Whites: the energetics study. , 2010, Journal of the American Dietetic Association.

[35]  P. Bajcsy,et al.  Recognition of arm gestures using multiple orientation sensors: gesture classification , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[36]  Abby G Ershow,et al.  An overview of methodologies, proficiencies, and training resources for controlled feeding studies. , 2003, Journal of the American Dietetic Association.

[37]  Michael R Neuman,et al.  Toward Objective Monitoring of Ingestive Behavior in Free‐living Population , 2009, Obesity.

[38]  C.J. De Luca,et al.  A Combined sEMG and Accelerometer System for Monitoring Functional Activity in Stroke , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[39]  T. Cole,et al.  Within- and between-subject variation in energy expenditure measured by the doubly-labelled water technique: implications for validating reported dietary energy intake , 2000, European Journal of Clinical Nutrition.

[40]  E. Finkelstein,et al.  Annual medical spending attributable to obesity: payer-and service-specific estimates. , 2009, Health affairs.

[41]  Edward Sazonov,et al.  Monitoring of Posture Allocations and Activities by a Shoe-Based Wearable Sensor , 2011, IEEE Transactions on Biomedical Engineering.

[42]  R. Wing,et al.  Long-term weight loss maintenance. , 2005, The American journal of clinical nutrition.

[43]  David S. Ebert,et al.  Technology-assisted dietary assessment , 2008, Electronic Imaging.

[44]  Edward Sazonov,et al.  Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior. , 2008, Physiological measurement.

[45]  T. Penney,et al.  Characterizing the obesogenic environment: the state of the evidence with directions for future research , 2010, Obesity reviews : an official journal of the International Association for the Study of Obesity.

[46]  Tom Baranowski,et al.  Need for technological innovation in dietary assessment. , 2010, Journal of the American Dietetic Association.

[47]  Kelly Wolfgang Is obesity a disease? , 2013, Advance for NPs & PAs.

[48]  M. Sevick,et al.  Self-monitoring in weight loss: a systematic review of the literature. , 2011, Journal of the American Dietetic Association.

[49]  E J Delp,et al.  Use of technology in children’s dietary assessment , 2009, European Journal of Clinical Nutrition.

[50]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[51]  B. McCabe-Sellers,et al.  Advancing the art and science of dietary assessment through technology. , 2010, Journal of the American Dietetic Association.

[52]  Steven K. Feiner,et al.  Exploring interaction with a simulated wrist-worn projection display , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[53]  O. Makeyev,et al.  Detection of Food Intake from Swallowing Sequences by Supervised and Unsupervised Methods , 2010, Annals of Biomedical Engineering.

[54]  Tae-Seong Kim,et al.  A Triaxial Accelerometer-Based Physical-Activity Recognition via Augmented-Signal Features and a Hierarchical Recognizer , 2010, IEEE Transactions on Information Technology in Biomedicine.

[55]  W. James International Association for the Study of Obesity and China , 2008, Obesity reviews : an official journal of the International Association for the Study of Obesity.

[56]  Miwako Doi,et al.  LifeMinder: a wearable healthcare support system using user's context , 2002, Proceedings 22nd International Conference on Distributed Computing Systems Workshops.

[57]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[58]  Corby K. Martin,et al.  Empirical evaluation of the ability to learn a calorie counting system and estimate portion size and food intake , 2007, British Journal of Nutrition.

[59]  J. Speakman,et al.  Doubly Labelled Water: Theory and Practice , 1997 .

[60]  Guang-Zhong Yang,et al.  Real-Time Activity Classification Using Ambient and Wearable Sensors , 2009, IEEE Transactions on Information Technology in Biomedicine.

[61]  G. Bray,et al.  Obesity as a Disease: A White Paper on Evidence and Arguments Commissioned by the Council of The Obesity Society , 2008, Obesity.

[62]  M. Wong,et al.  Epidemiological assessment of diet: a comparison of a 7-day diary with a food frequency questionnaire using urinary markers of nitrogen, potassium and sodium. , 2001, International journal of epidemiology.

[63]  Michael R. Neuman,et al.  Automatic Detection of Swallowing Events by Acoustical Means for Applications of Monitoring of Ingestive Behavior , 2010, IEEE Transactions on Biomedical Engineering.

[64]  Jane Yung-jen Hsu,et al.  The Diet-Aware Dining Table: Observing Dietary Behaviors over a Tabletop Surface , 2006, Pervasive.

[65]  R. Groff AN ASSESSMENT OF LINEAR WRIST MOTION DURING THE TAKING OF A BITE OF FOOD , 2010 .

[66]  R. Wing,et al.  Successful weight loss maintenance. , 2003, Annual review of nutrition.

[67]  Juha Pärkkä,et al.  Personalization Algorithm for Real-Time Activity Recognition Using PDA, Wireless Motion Bands, and Binary Decision Tree , 2010, IEEE Transactions on Information Technology in Biomedicine.

[68]  Yoshihiro Saeki,et al.  Proposal of Food Intake Measuring System in Medical Use and Its Discussion of Practical Capability , 2005, KES.

[69]  Hassan Ghasemzadeh,et al.  Structural Action Recognition in Body Sensor Networks: Distributed Classification Based on String Matching , 2010, IEEE Transactions on Information Technology in Biomedicine.

[70]  Paul Lukowicz,et al.  Using ultrasonic hand tracking to augment motion analysis based recognition of manipulative gestures , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[71]  Merryn J Mathie,et al.  Accelerometry: providing an integrated, practical method for long-term, ambulatory monitoring of human movement , 2004, Physiological measurement.

[72]  S. Thuillier,et al.  Fibre optic sensor for physiological parameters , 1998 .

[73]  Gerhard Tröster,et al.  On-Body Sensing Solutions for Automatic Dietary Monitoring , 2009, IEEE Pervasive Computing.

[74]  B. Dennis Well-controlled diet studies in humans , 1999 .

[75]  P. Blackman Actual causes of death in the United States. , 1994, JAMA.

[76]  Yujie Dong,et al.  Measuring Digital System Latency from Sensing to Actuation at Continuous 1-ms Resolution , 2013, PRESENCE: Teleoperators and Virtual Environments.

[77]  D. Schoeller,et al.  Evaluation of dietary assessment instruments against doubly labeled water, a biomarker of habitual energy intake. , 2001, American journal of physiology. Endocrinology and metabolism.

[78]  J. Scisco The Bite Detector: A Device for the Behavioral Treatment of Overweight and Obesity , 2009 .

[79]  Usability and Acceptability of the “Bite Counter” Device , 2011 .

[80]  Adam W. Hoover,et al.  A New Method for Measuring Meal Intake in Humans via Automated Wrist Motion Tracking , 2012, Applied Psychophysiology and Biofeedback.

[81]  J. Brug,et al.  Are awareness of dietary fat intake and actual fat consumption associated?—A Dutch--American Comparison , 1997, European Journal of Clinical Nutrition.

[82]  R. Pool Fat: Fighting the Obesity Epidemic , 2001 .

[83]  C. Champagne,et al.  Energy intake and energy expenditure: a controlled study comparing dietitians and non-dietitians. , 2002, Journal of the American Dietetic Association.

[84]  Paul Lukowicz,et al.  Gesture spotting with body-worn inertial sensors to detect user activities , 2008, Pattern Recognit..

[85]  M. Krawczak,et al.  Genetic studies of common types of obesity: a critique of the current use of phenotypes , 2010, Obesity reviews : an official journal of the International Association for the Study of Obesity.

[86]  J. Scisco,et al.  An environmental cue to eat slowly reduces food consumption , 2010, Appetite.

[87]  Ilkka Korhonen,et al.  Detection of Daily Activities and Sports With Wearable Sensors in Controlled and Uncontrolled Conditions , 2008, IEEE Transactions on Information Technology in Biomedicine.

[88]  L. Terre,et al.  Overview and the Future of Obesity Treatment , 2005 .

[89]  Detecting Eating Using a Wrist Mounted Device During Normal Daily Activities , 2011 .

[90]  Daniel P Heil,et al.  Characterizing free-living light exposure using a wrist-worn light monitor. , 2002, Applied ergonomics.

[91]  Jeannette Beasley,et al.  Accuracy of a PDA-based dietary assessment program. , 2005, Nutrition.

[92]  Clare E Collins,et al.  A systematic review of the validity of dietary assessment methods in children when compared with the method of doubly labeled water. , 2010, Journal of the American Dietetic Association.

[93]  Qiang Yang,et al.  Sensor-Based Abnormal Human-Activity Detection , 2008, IEEE Transactions on Knowledge and Data Engineering.

[94]  Yujie Dong,et al.  A Device for Detecting and Counting Bites of Food Taken by a Person during Eating , 2009, IEEE International Conference on Bioinformatics and Biomedicine.

[95]  David S. Ebert,et al.  The Use of Mobile Devices in Aiding Dietary Assessment and Evaluation , 2010, IEEE Journal of Selected Topics in Signal Processing.

[96]  N. Wellman,et al.  Causes and consequences of adult obesity: health, social and economic impacts in the United States , 2002 .

[97]  S. Heymsfield,et al.  Do unsuccessful dieters intentionally underreport food intake? , 1998, The International journal of eating disorders.

[98]  S. Schuckers,et al.  The Energetics of Obesity: A Review: Monitoring Energy Intake and Energy Expenditure in Humans , 2010, IEEE Engineering in Medicine and Biology Magazine.

[99]  M. N. Husain,et al.  Fitness monitor system , 2003, TENCON 2003. Conference on Convergent Technologies for Asia-Pacific Region.