A Mobile-Based Diet Monitoring System for Obesity Management.

Personal diet management is key to fighting the obesity epidemic. Recent advances in smartphones and wearable sensor technologies have empowered automated food monitoring through food image processing and eating episode detection, with the goal to conquer drawbacks of traditional food journaling that is labour intensive, inaccurate, and low adherent. In this paper, we present a new interactive mobile system that enables automated food recognition and assessment based on user food images and provides dietary intervention while tracking users' dietary and physical activities. In addition to using techniques in computer vision and machine learning, one unique feature of this system is the realization of real-time energy balance monitoring through metabolic network simulation. As a proof of concept, we have demonstrated the use of this system through an Android application.

[1]  Adam W. Hoover,et al.  A New Method for Measuring Meal Intake in Humans via Automated Wrist Motion Tracking , 2012, Applied Psychophysiology and Biofeedback.

[2]  J. Rabinowitz,et al.  Absolute Metabolite Concentrations and Implied Enzyme Active Site Occupancy in Escherichia coli , 2009, Nature chemical biology.

[3]  Ming Ouhyoung,et al.  Automatic Chinese food identification and quantity estimation , 2012, SIGGRAPH Asia Technical Briefs.

[4]  Keiji Yanai,et al.  Automatic Expansion of a Food Image Dataset Leveraging Existing Categories with Domain Adaptation , 2014, ECCV Workshops.

[5]  Sergio Guadarrama,et al.  Im2Calories: Towards an Automated Mobile Vision Food Diary , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[6]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[8]  Matthieu Guillaumin,et al.  Food-101 - Mining Discriminative Components with Random Forests , 2014, ECCV.

[9]  Keiji Yanai,et al.  Real-Time Mobile Food Recognition System , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[10]  Hiroyuki Ogata,et al.  KEGG: Kyoto Encyclopedia of Genes and Genomes , 1999, Nucleic Acids Res..

[11]  Keiji Yanai,et al.  Recognition of Multiple-Food Images by Detecting Candidate Regions , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[12]  N. Ranganathan,et al.  Gabor filter-based edge detection , 1992, Pattern Recognit..

[13]  E. Fehlberg,et al.  Low-order classical Runge-Kutta formulas with stepsize control and their application to some heat transfer problems , 1969 .

[14]  William M. Campbell,et al.  Support vector machines for speaker verification and identification , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[15]  Janine L. Wright,et al.  The connecting health and technology study: a 6-month randomized controlled trial to improve nutrition behaviours using a mobile food record and text messaging support in young adults , 2016, International Journal of Behavioral Nutrition and Physical Activity.

[16]  Jungmin Chung,et al.  A glasses-type wearable device for monitoring the patterns of food intake and facial activity , 2017, Scientific Reports.

[17]  Massimiliano Pierobon,et al.  A simulation model of glucose-insulin metabolism and implementation on OSG , 2017, 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[18]  Jerilyn K Allen,et al.  Randomized Controlled Pilot Study Testing Use of Smartphone Technology for Obesity Treatment , 2013, Journal of obesity.

[19]  Keiji Yanai,et al.  Food image recognition using deep convolutional network with pre-training and fine-tuning , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[20]  Vieira Bruno,et al.  A Survey on Automated Food Monitoring and Dietary Management Systems , 2017, Journal of health & medical informatics.

[21]  E. Delp,et al.  How Willing Are Adolescents to Record Their Dietary Intake? The Mobile Food Record , 2015, JMIR mHealth and uHealth.

[22]  Oliver Amft,et al.  Ambient, On-Body, and Implantable Monitoring Technologies to Assess Dietary Behavior , 2011 .

[23]  Paolo Napoletano,et al.  Food Recognition: A New Dataset, Experiments, and Results , 2017, IEEE Journal of Biomedical and Health Informatics.

[24]  Mei Chen,et al.  Food recognition using statistics of pairwise local features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Ying Zhang,et al.  HMDB: the Human Metabolome Database , 2007, Nucleic Acids Res..

[26]  Edward Sazonov,et al.  A Novel Wearable Device for Food Intake and Physical Activity Recognition , 2016, Sensors.

[27]  Marios Anthimopoulos,et al.  A Food Recognition System for Diabetic Patients Based on an Optimized Bag-of-Features Model , 2014, IEEE Journal of Biomedical and Health Informatics.

[28]  T. Lobstein,et al.  Defining and labelling ‘healthy’ and ‘unhealthy’ food , 2008, Public Health Nutrition.

[29]  Neel Joshi,et al.  Menu-Match: Restaurant-Specific Food Logging from Images , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[30]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[31]  Max Mignotte,et al.  Segmentation by Fusion of Histogram-Based $K$-Means Clusters in Different Color Spaces , 2008, IEEE Transactions on Image Processing.

[32]  Abdulsalam Yassine,et al.  Using graph cut segmentation for food calorie measurement , 2014, 2014 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[33]  Wen-Hui Chen,et al.  Intelligent ontological agent for diabetic food recommendation , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[34]  Douglas E Levy,et al.  Traffic-light labels and choice architecture: promoting healthy food choices. , 2014, American journal of preventive medicine.

[35]  M. Carter,et al.  Adherence to a Smartphone Application for Weight Loss Compared to Website and Paper Diary: Pilot Randomized Controlled Trial , 2013, Journal of medical Internet research.

[36]  C. Champagne,et al.  Energy intake and energy expenditure: a controlled study comparing dietitians and non-dietitians. , 2002, Journal of the American Dietetic Association.

[37]  Sharifullah Khan,et al.  An integrated approach of diet and exercise recommendations for diabetes patients , 2014, 2014 IEEE 16th International Conference on e-Health Networking, Applications and Services (Healthcom).

[38]  Edward J. Delp,et al.  Multiple Hypotheses Image Segmentation and Classification With Application to Dietary Assessment , 2015, IEEE Journal of Biomedical and Health Informatics.

[39]  Lei Yang,et al.  PFID: Pittsburgh fast-food image dataset , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[40]  Gregory D. Abowd,et al.  Barriers and Negative Nudges: Exploring Challenges in Food Journaling , 2015, CHI.

[41]  Mudita Singhal,et al.  COPASI - a COmplex PAthway SImulator , 2006, Bioinform..

[42]  Krzysztof Z. Gajos,et al.  Platemate: crowdsourcing nutritional analysis from food photographs , 2011, UIST.

[43]  Chong-Wah Ngo,et al.  Deep-based Ingredient Recognition for Cooking Recipe Retrieval , 2016, ACM Multimedia.

[44]  Monica Mordonini,et al.  Food Image Recognition Using Very Deep Convolutional Networks , 2016, MADiMa @ ACM Multimedia.

[45]  Edward J. Delp,et al.  Analysis of food images: Features and classification , 2014, 2014 IEEE International Conference on Image Processing (ICIP).