Computer Vision-Based Carbohydrate Estimation for Type 1 Patients With Diabetes Using Smartphones

Background: Individuals with type 1 diabetes (T1D) have to count the carbohydrates (CHOs) of their meal to estimate the prandial insulin dose needed to compensate for the meal’s effect on blood glucose levels. CHO counting is very challenging but also crucial, since an error of 20 grams can substantially impair postprandial control. Method: The GoCARB system is a smartphone application designed to support T1D patients with CHO counting of nonpacked foods. In a typical scenario, the user places a reference card next to the dish and acquires 2 images with his/her smartphone. From these images, the plate is detected and the different food items on the plate are automatically segmented and recognized, while their 3D shape is reconstructed. Finally, the food volumes are calculated and the CHO content is estimated by combining the previous results and using the USDA nutritional database. Results: To evaluate the proposed system, a set of 24 multi-food dishes was used. For each dish, 3 pairs of images were taken and for each pair, the system was applied 4 times. The mean absolute percentage error in CHO estimation was 10 ± 12%, which led to a mean absolute error of 6 ± 8 CHO grams for normal-sized dishes. Conclusion: The laboratory experiments demonstrated the feasibility of the GoCARB prototype system since the error was below the initial goal of 20 grams. However, further improvements and evaluation are needed prior launching a system able to meet the inter- and intracultural eating habits.

[1]  M. Allman-Farinelli,et al.  Feasibility and validity of mobile phones to assess dietary intake. , 2014, Nutrition.

[2]  J. Edge,et al.  Can children with Type 1 diabetes and their caregivers estimate the carbohydrate content of meals and snacks? , 2009, Diabetic medicine : a journal of the British Diabetic Association.

[3]  V. Basevi.  Standards of Medical Care in Diabetes—2010 , 2010, Diabetes Care.

[4]  Ransford Henry Botwey,et al.  A computer vision-based Smartphone system for carbohydrate counting , 2015 .

[5]  Lei Yang,et al.  PFID: Pittsburgh fast-food image dataset , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[6]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[7]  P. McElduff,et al.  In children using intensive insulin therapy, a 20‐g variation in carbohydrate amount significantly impacts on postprandial glycaemia , 2012, Diabetic medicine : a journal of the British Diabetic Association.

[8]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[9]  Jindong Tan,et al.  DietCam: Automatic dietary assessment with mobile camera phones , 2012, Pervasive Mob. Comput..

[10]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[11]  Aesha Drozdowski,et al.  Standards of medical care in diabetes. , 2004, Diabetes care.

[12]  David V. Power,et al.  Standards of Medical Care in Diabetes: Response to position statement of the American Diabetes Association , 2006 .

[13]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Daniel P. Siewiorek,et al.  Wearable context-aware food recognition for calorie monitoring , 2008, 2008 12th IEEE International Symposium on Wearable Computers.

[15]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[16]  Nicholas Chen,et al.  Toward Dietary Assessment via Mobile Phone Video Cameras. , 2010, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[17]  J. Edge,et al.  Children and adolescents on intensive insulin therapy maintain postprandial glycaemic control without precise carbohydrate counting , 2009, Diabetic medicine : a journal of the British Diabetic Association.

[18]  Wen Wu,et al.  Fast food recognition from videos of eating for calorie estimation , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[19]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[20]  Zhiwei Zhu,et al.  Recognition and volume estimation of food intake using a mobile device , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[21]  A. Brazeau,et al.  Carbohydrate counting accuracy and blood glucose variability in adults with type 1 diabetes. , 2013, Diabetes research and clinical practice.

[22]  Marios Anthimopoulos,et al.  Segmentation and recognition of multi-food meal images for carbohydrate counting , 2013, 13th IEEE International Conference on BioInformatics and BioEngineering.

[23]  Stavroula G. Mougiakakou,et al.  Food volume computation for self dietary assessment applications , 2013, 13th IEEE International Conference on BioInformatics and BioEngineering.

[24]  D. Maahs,et al.  The Carbohydrate Counting in Adolescents With Type 1 Diabetes (CCAT) Study , 2009 .

[25]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[26]  Reinhard Koch,et al.  A simple and efficient rectification method for general motion , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[27]  M. S. Kirkman,et al.  Response to Comment on: American Diabetes Association. Standards of Medical Care in Diabetes—2011. Diabetes Care 2011;34(Suppl. 1):S11–S61 , 2011, Diabetes Care.

[28]  Shervin Shirmohammadi,et al.  Intelligent SVM based food intake measurement system , 2013, 2013 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA).

[29]  David S. Ebert,et al.  The Use of Mobile Devices in Aiding Dietary Assessment and Evaluation , 2010, IEEE Journal of Selected Topics in Signal Processing.

[30]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.