Vision-Based Food Analysis for Automatic Dietary Assessment

aSchool of Computer and Engineering, Beijing Technology and Business University, Beijing, 100048, China bThe Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China cBeijing Key Laboratory of Big Data Technology for Food Safety, Beijing, 100048, China dNational Engineering Laboratory For Agri-product Quality Traceability, Beijing, 100048, China eUniversity of Chinese Academy of Sciences, Beijing, 100049, China

[1]  Zhiling Wang,et al.  ISIA Food-500: A Dataset for Large-Scale Food Recognition via Stacked Global-Local Attention Network , 2020, ACM Multimedia.

[2]  Yingnan Sun,et al.  Point2Volume: A Vision-Based Dietary Assessment Approach Using View Synthesis , 2020, IEEE Transactions on Industrial Informatics.

[3]  Shuqiang Jiang,et al.  Large Scale Visual Food Recognition , 2021, ArXiv.

[4]  Benny P. L. Lo,et al.  Food volume estimation for quantifying dietary intake with a wearable camera , 2018, 2018 IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[5]  J. Hebebrand,et al.  [Obesity and overweight]. , 2009, Zeitschrift fur Kinder- und Jugendpsychiatrie und Psychotherapie.

[6]  Neel Joshi,et al.  Menu-Match: Restaurant-Specific Food Logging from Images , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[7]  Thomai Stathopoulou,et al.  goFOODTM: An Artificial Intelligence System for Dietary Assessment , 2020, Sensors.

[8]  Edward J. Delp,et al.  Image-based food volume estimation , 2013, CEA '13.

[9]  Qiang Yang,et al.  An Overview of Multi-task Learning , 2018 .

[10]  Armin Lawi,et al.  Food Constituent Estimation for Lifestyle Disease Prevention by Multi-Task CNN , 2019, Appl. Artif. Intell..

[11]  Sebastian Ruder,et al.  An Overview of Multi-Task Learning in Deep Neural Networks , 2017, ArXiv.

[12]  Fahad Shahbaz Khan,et al.  Transformers in Vision: A Survey , 2021, ACM Comput. Surv..

[13]  Stergios Christodoulidis,et al.  An Artificial Intelligence-Based System to Assess Nutrient Intake for Hospitalised Patients , 2020, IEEE Transactions on Multimedia.

[14]  Keiji Yanai,et al.  Multi-task learning of dish detection and calorie estimation , 2018, MADiMa@IJCAI.

[15]  Betty P. Perloff,et al.  USDA Food and Nutrient Database for Dietary Studies : Released on the web , 2006 .

[16]  Kiyoharu Aizawa,et al.  Food log by analyzing food images , 2008, ACM Multimedia.

[17]  Daniel P. Siewiorek,et al.  Wearable context-aware food recognition for calorie monitoring , 2008, 2008 12th IEEE International Symposium on Wearable Computers.

[18]  Keiji Yanai,et al.  Image-Based Food Calorie Estimation Using Knowledge on Food Categories, Ingredients and Cooking Directions , 2017, ACM Multimedia.

[19]  Paolo Napoletano,et al.  Food Recognition and Leftover Estimation for Daily Diet Monitoring , 2015, ICIAP Workshops.

[20]  Rainer Stiefelhagen,et al.  Multi-Task Learning for Calorie Prediction on a Novel Large-Scale Recipe Dataset Enriched with Nutritional Information , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[21]  Arjun Karpur,et al.  Nutrition5k: Towards Automatic Nutritional Understanding of Generic Food , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Fengqing Zhu,et al.  Towards Learning Food Portion From Monocular Images With Cross-Domain Feature Adaptation , 2021, 2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP).

[23]  Marios Anthimopoulos,et al.  Dish Detection and Segmentation for Dietary Assessment on Smartphones , 2015, ICIAP Workshops.

[24]  Mingui Sun,et al.  Model-based measurement of food portion size for image-based dietary assessment using 3D/2D registration , 2013, Measurement science & technology.

[25]  Edward J. Delp,et al.  Multiple Hypotheses Image Segmentation and Classification With Application to Dietary Assessment , 2015, IEEE Journal of Biomedical and Health Informatics.

[26]  Mark R. Pickering,et al.  Food Volume Estimation in a Mobile Phone Based Dietary Assessment System , 2012, 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems.

[27]  Fengqing Zhu,et al.  An End-to-End Food Image Analysis System , 2021, Electronic Imaging.

[28]  Ramesh C. Jain,et al.  A Survey on Food Computing , 2018, ACM Comput. Surv..

[29]  I. Huybrechts,et al.  Review and evaluation of innovative technologies for measuring diet in nutritional epidemiology. , 2012, International journal of epidemiology.

[30]  Edward J. Delp,et al.  Single-View Food Portion Estimation: Learning Image-to-Energy Mappings Using Generative Adversarial Networks , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[31]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Pei-Yu Chi,et al.  Enabling Calorie-Aware Cooking in a Smart Kitchen , 2008, PERSUASIVE.

[33]  Zhiwei Zhu,et al.  Recognition and volume estimation of food intake using a mobile device , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[34]  Hongbin Pu,et al.  Efficient extraction of deep image features using convolutional neural network (CNN) for applications in detecting and analysing complex food matrices , 2021, Trends in Food Science & Technology.

[35]  Wen Wu,et al.  Fast food recognition from videos of eating for calorie estimation , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[36]  Marios Anthimopoulos,et al.  Two-View 3D Reconstruction for Food Volume Estimation , 2017, IEEE Transactions on Multimedia.

[37]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Ming Ouhyoung,et al.  Automatic Chinese food identification and quantity estimation , 2012, SIGGRAPH Asia Technical Briefs.

[40]  Stavroula G. Mougiakakou,et al.  Self-Attention and Ingredient-Attention Based Model for Recipe Retrieval from Image Queries , 2019, MADiMa @ ACM Multimedia.

[41]  Abdulsalam Yassine,et al.  Food calorie measurement using deep learning neural network , 2016, 2016 IEEE International Instrumentation and Measurement Technology Conference Proceedings.

[42]  Thomai Stathopoulou,et al.  Partially Supervised Multi-Task Network for Single-View Dietary Assessment , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[43]  Mingui Sun,et al.  Image-based food portion size estimation using a smartphone without a fiducial marker , 2018, Public Health Nutrition.

[44]  Julien Delarue,et al.  Dynamics of food preferences: a case study with chewing gums , 2004 .

[45]  D. McDonald,et al.  Personalized nutrition through big data , 2016, Nature Biotechnology.

[46]  Vinod Vokkarane,et al.  A New Deep Learning-Based Food Recognition System for Dietary Assessment on An Edge Computing Service Infrastructure , 2018, IEEE Transactions on Services Computing.

[47]  Shervin Shirmohammadi,et al.  Measuring Calorie and Nutrition From Food Image , 2014, IEEE Transactions on Instrumentation and Measurement.

[48]  Hui Zhang,et al.  Camera Calibration from Images of Spheres , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Alex Mihailidis,et al.  An Intelligent Nutritional Assessment System , 2012, AAAI Fall Symposium: Artificial Intelligence for Gerontechnology.

[50]  Kiyoharu Aizawa,et al.  Image-based Calorie Content Estimation for Dietary Assessment , 2011, 2011 IEEE International Symposium on Multimedia.

[51]  Shervin Shirmohammadi,et al.  Mobile Multi-Food Recognition Using Deep Learning , 2017, ACM Trans. Multim. Comput. Commun. Appl..

[52]  Jindong Tan,et al.  DietCam: Automatic dietary assessment with mobile camera phones , 2012, Pervasive Mob. Comput..

[53]  Bhupinder Kaur,et al.  Current Developments in Digital Quantitative Volume Estimation for the Optimisation of Dietary Assessment , 2020, Nutrients.

[54]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[55]  Chun Pong Lau,et al.  Image-based nutrient estimation for Chinese dishes using deep learning. , 2021, Food research international.

[56]  Håkan Jönsson,et al.  Food and health: individual, cultural, or scientific matters? , 2013, Genes & Nutrition.

[57]  Keiji Yanai,et al.  Simultaneous Estimation of Dish Locations and Calories with Multi-Task Learning , 2019, IEICE Trans. Inf. Syst..

[58]  Antonio J. Plaza,et al.  Image Segmentation Using Deep Learning: A Survey , 2021, IEEE transactions on pattern analysis and machine intelligence.

[59]  Wanqing Li,et al.  Dietary Assessment on a Mobile Phone Using Image Processing and Pattern Recognition Techniques: Algorithm Design and System Prototyping , 2015, Nutrients.

[60]  David S. Ebert,et al.  Technology-assisted dietary assessment , 2008, Electronic Imaging.

[61]  Marios Anthimopoulos,et al.  Food Recognition for Dietary Assessment Using Deep Convolutional Neural Networks , 2015, ICIAP Workshops.

[62]  David S. Ebert,et al.  Automatic portion estimation and visual refinement in mobile dietary assessment , 2010, Electronic Imaging.

[63]  Abdulsalam Yassine,et al.  Using graph cut segmentation for food calorie measurement , 2014, 2014 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[64]  Kyungwon Oh,et al.  Dietary assessment methods in epidemiologic studies , 2014, Epidemiology and health.

[65]  Deanna M. Hoelscher,et al.  Dietary Assessment Methods among School-Aged Children: Validity and Reliability , 2000 .

[66]  Mohammed Ahmed Subhi,et al.  Vision-Based Approaches for Automatic Food Recognition and Dietary Assessment: A Survey , 2019, IEEE Access.

[67]  Keiji Yanai,et al.  GrillCam: A Real-Time Eating Action Recognition System , 2016, MMM.

[68]  David S. Ebert,et al.  The Use of Mobile Devices in Aiding Dietary Assessment and Evaluation , 2010, IEEE Journal of Selected Topics in Signal Processing.

[69]  D. Allison,et al.  Uncertainty in human nutrition research , 2020, Nature Food.

[70]  Benny Lo,et al.  Assessing Individual Dietary Intake in Food Sharing Scenarios with a 360 Camera and Deep Learning , 2019, 2019 IEEE 16th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[71]  Yingnan Sun,et al.  Image-Based Food Classification and Volume Estimation for Dietary Assessment: A Review , 2020, IEEE Journal of Biomedical and Health Informatics.

[72]  Gwen L. Alexander,et al.  Comparison of Interviewer-Administered and Automated Self-Administered 24-Hour Dietary Recalls in 3 Diverse Integrated Health Systems. , 2015, American journal of epidemiology.

[73]  Shady Elbassuoni,et al.  Calories Prediction from Food Images , 2017, AAAI.

[74]  Abdulsalam Yassine,et al.  FooDD: Food Detection Dataset for Calorie Measurement Using Food Images , 2015, ICIAP Workshops.

[75]  Luka Šajn,et al.  Food object recognition using a mobile device: Evaluation of currently implemented systems , 2020 .

[76]  Giovanni Maria Farinella,et al.  A Multimedia Database for Automatic Meal Assessment Systems , 2017, ICIAP Workshops.

[77]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[78]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[79]  Wenjun Zeng,et al.  S2R-DepthNet: Learning a Generalizable Depth-specific Structural Representation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[80]  Marios Anthimopoulos,et al.  Food Image Segmentation for Dietary Assessment , 2016, MADiMa @ ACM Multimedia.

[81]  Fengqing Zhu,et al.  Multi-task Image-Based Dietary Assessment for Food Recognition and Portion Size Estimation , 2020, 2020 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR).

[82]  Keiji Yanai,et al.  An Automatic Calorie Estimation System of Food Images on a Smartphone , 2016, MADiMa @ ACM Multimedia.

[83]  R. Wells,et al.  A future workforce of food-system analysts , 2019, Nature Food.

[84]  Qingjin Peng,et al.  Robust recognition of checkerboard pattern for camera calibration , 2006 .

[85]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[86]  David S. Ebert,et al.  Personal dietary assessment using mobile devices , 2009, Electronic Imaging.

[87]  Edward J. Delp,et al.  An image analysis system for dietary assessment and evaluation , 2010, 2010 IEEE International Conference on Image Processing.

[88]  Giovanni Maria Farinella,et al.  A multi-task learning approach for meal assessment , 2018, MADiMa@IJCAI.

[89]  Yukinobu Taniguchi,et al.  Estimating nutritional value from food images based on semantic segmentation , 2014, UbiComp Adjunct.

[90]  Fengqing Zhu,et al.  An End-to-End Image-Based Automatic Food Energy Estimation Technique Based on Learned Energy Distribution Images: Protocol and Methodology , 2019, Nutrients.

[91]  Marios Anthimopoulos,et al.  Computer Vision-Based Carbohydrate Estimation for Type 1 Patients With Diabetes Using Smartphones , 2015, Journal of diabetes science and technology.

[92]  Wei Chen,et al.  Dietary patterns affect Parkinson's disease via the microbiota-gut-brain axis , 2021 .

[93]  Edward J. Delp,et al.  Single-View Food Portion Estimation Based on Geometric Models , 2015, 2015 IEEE International Symposium on Multimedia (ISM).

[94]  Chu Kiong Loo,et al.  A Review of the Vision-based Approaches for Dietary Assessment , 2021, ArXiv.

[95]  Edward J. Delp,et al.  Food image analysis: Segmentation, identification and weight estimation , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[96]  Sergio Guadarrama,et al.  Im2Calories: Towards an Automated Mobile Vision Food Diary , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[97]  Ramesh Jain,et al.  Food Recommendation: Framework, Existing Solutions, and Challenges , 2019, IEEE Transactions on Multimedia.

[98]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[99]  B. Koroušić Seljak,et al.  NutriNet: A Deep Learning Food and Drink Image Recognition System for Dietary Assessment , 2017, Nutrients.