Food Recognition for Dietary Assessment Using Deep Convolutional Neural Networks

Diet management is a key factor for the prevention and treatment of diet-related chronic diseases. Computer vision systems aim to provide automated food intake assessment using meal images. We propose a method for the recognition of already segmented food items in meal images. The method uses a 6-layer deep convolutional neural network to classify food image patches. For each food item, overlapping patches are extracted and classified and the class with the majority of votes is assigned to it. Experiments on a manually annotated dataset with 573 food items justified the choice of the involved components and proved the effectiveness of the proposed system yielding an overall accuracy of 84.9%.

[1]  Neel Joshi,et al.  Menu-Match: Restaurant-Specific Food Logging from Images , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[2]  Wanqing Li,et al.  Food image classification using local appearance and global structural information , 2014, Neurocomputing.

[3]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[4]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[5]  Zhiwei Zhu,et al.  Recognition and volume estimation of food intake using a mobile device , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[6]  Gregory D. Abowd,et al.  Leveraging Context to Support Automated Food Recognition in Restaurants , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[7]  Mei Chen,et al.  Food recognition using statistics of pairwise local features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Makoto Ogawa,et al.  Food Detection and Recognition Using Convolutional Neural Network , 2014, ACM Multimedia.

[9]  Lei Yang,et al.  PFID: Pittsburgh fast-food image dataset , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[10]  Giovanni Maria Farinella,et al.  Classifying food images represented as Bag of Textons , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[11]  Marios Anthimopoulos,et al.  A Food Recognition System for Diabetic Patients Based on an Optimized Bag-of-Features Model , 2014, IEEE Journal of Biomedical and Health Informatics.

[12]  Edward J. Delp,et al.  Multiple Hypotheses Image Segmentation and Classification With Application to Dietary Assessment , 2015, IEEE Journal of Biomedical and Health Informatics.

[13]  Keiji Yanai,et al.  FoodCam: A Real-Time Mobile Food Recognition System Employing Fisher Vector , 2014, MMM.

[14]  Keiji Yanai,et al.  FoodCam: A real-time food recognition system on a smartphone , 2015, Multimedia Tools and Applications.

[15]  Marios Anthimopoulos,et al.  Segmentation and recognition of multi-food meal images for carbohydrate counting , 2013, 13th IEEE International Conference on BioInformatics and BioEngineering.

[16]  Keiji Yanai,et al.  Recognition of Multiple-Food Images by Detecting Candidate Regions , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[17]  Ming Ouhyoung,et al.  Automatic Chinese food identification and quantity estimation , 2012, SIGGRAPH Asia Technical Briefs.

[18]  Kiyoharu Aizawa,et al.  Food Balance Estimation by Using Personal Dietary Tendencies in a Multimedia Food Log , 2013, IEEE Transactions on Multimedia.

[19]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[20]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[21]  Talmai Oliveira,et al.  A mobile, lightweight, poll-based food identification system , 2014, Pattern Recognit..

[22]  Matthieu Guillaumin,et al.  Food-101 - Mining Discriminative Components with Random Forests , 2014, ECCV.

[23]  Keiji Yanai,et al.  Food image recognition with deep convolutional features , 2014, UbiComp Adjunct.