The use of co-occurrence patterns in single image based food portion estimation

Measuring accurate dietary intake is considered to be an open research problem in the nutrition and health fields. Food portions estimation is a challenging problem as food preparation and consumption process pose large variations on food shapes and appearances. We use geometric model based technique to estimate food portions and further improve estimation accuracy using co-occurrence patterns. We estimate the food portion co-occurrence patterns from food images we collected from dietary studies using the mobile Food Record (mFR) system we developed. Co-occurrence patterns is used as prior knowledge to refine portion estimation results. We show that the portion estimation accuracy has been improved when in-corporating the co-occurrence patterns as contextual information.

[1]  Mingui Sun,et al.  3D/2D model-to-image registration for quantitative dietary assessment , 2012, 2012 38th Annual Northeast Bioengineering Conference (NEBEC).

[2]  Edward J. Delp,et al.  A comparison of food portion size estimation using geometric models and depth images , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[3]  Sai Siva Gorthi,et al.  Fringe projection techniques: Whither we are? , 2010 .

[4]  Edward J. Delp,et al.  Feasibility and Use of the Mobile Food Record for Capturing Eating Occasions among Children Ages 3–10 Years in Guam , 2015, Nutrients.

[5]  Rob Fergus,et al.  Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[6]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[7]  Edward J. Delp,et al.  Context based food image analysis , 2013, 2013 IEEE International Conference on Image Processing.

[8]  Ming Ouhyoung,et al.  Automatic Chinese food identification and quantity estimation , 2012, SIGGRAPH Asia Technical Briefs.

[9]  Shervin Shirmohammadi,et al.  Measuring Calorie and Nutrition From Food Image , 2014, IEEE Transactions on Instrumentation and Measurement.

[10]  David S. Ebert,et al.  The Use of Mobile Devices in Aiding Dietary Assessment and Evaluation , 2010, IEEE Journal of Selected Topics in Signal Processing.

[11]  Edward J. Delp,et al.  Image-based food volume estimation , 2013, CEA '13.

[12]  Edward J. Delp,et al.  Analysis of food images: Features and classification , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[13]  Edward J. Delp,et al.  Single-View Food Portion Estimation Based on Geometric Models , 2015, 2015 IEEE International Symposium on Multimedia (ISM).

[14]  Kiyoharu Aizawa,et al.  Food Balance Estimation by Using Personal Dietary Tendencies in a Multimedia Food Log , 2013, IEEE Transactions on Multimedia.

[15]  E. Delp,et al.  Comparison of Known Food Weights with Image-Based Portion-Size Automated Estimation and Adolescents' Self-Reported Portion Size , 2012, Journal of diabetes science and technology.

[16]  E. Delp,et al.  Novel Technologies for Assessing Dietary Intake: Evaluating the Usability of a Mobile Telephone Food Record Among Adults and Adolescents , 2012, Journal of medical Internet research.

[17]  Gary R. Bradski,et al.  Learning OpenCV - computer vision with the OpenCV library: software that sees , 2008 .

[18]  Mingui Sun,et al.  A wearable electronic system for objective dietary assessment. , 2010, Journal of the American Dietetic Association.

[19]  Zhiwei Zhu,et al.  Recognition and volume estimation of food intake using a mobile device , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[20]  Edward J. Delp,et al.  Multiple Hypotheses Image Segmentation and Classification With Application to Dietary Assessment , 2015, IEEE Journal of Biomedical and Health Informatics.

[21]  E J Delp,et al.  Use of technology in children’s dietary assessment , 2009, European Journal of Clinical Nutrition.

[22]  Keiji Yanai,et al.  A food image recognition system with Multiple Kernel Learning , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[23]  E. Delp,et al.  Evidence-based development of a mobile telephone food record. , 2010, Journal of the American Dietetic Association.

[24]  Xing Zhang,et al.  A mobile structured light system for food volume estimation , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[25]  Behjat Siddiquie,et al.  “Snap-n-Eat” , 2015, Journal of diabetes science and technology.

[26]  Jindong Tan,et al.  DietCam: Automatic dietary assessment with mobile camera phones , 2012, Pervasive Mob. Comput..

[27]  Sergio Guadarrama,et al.  Im2Calories: Towards an Automated Mobile Vision Food Diary , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[28]  Kiyoharu Aizawa,et al.  FoodLog: capture, analysis and retrieval of personal food images via web , 2009, CEA '09.