Use of smartphone videos and pattern recognition for food authentication

Abstract A novel sensor system for food authentication is presented, which is based on computer vision and pattern recognition. The sensor system uses a smartphone to generate a sequence of light with varying colours to illuminate a food sample, and uses the smartphone camera to receive reflected light by way of recording a video. The video is processed using computer vision techniques and transformed into sensor data in the form of a data vector. The sensor data is analysed using pattern recognition techniques. The locally weighted partial least squares regression method is extended for classification to improve the modelling effectiveness and robustness. The sensor system is evaluated on the task of authentication of olive oil and milk – to verify how they are labelled. Large quantities of olive oil and milk were purchased from supermarkets, and sensor videos were created using the sensor system. Test accuracies of 96.2% and 100% were achieved for olive oil and milk authentication respectively. These results suggest the proposed sensor system is effective. Since the sensor system is built in a smartphone, it has the potential to serve as a low-cost and effective solution for food authentication and to empower consumers in food fraud detection.

[1]  Ronald D. Snee,et al.  Validation of Regression Models: Methods and Examples , 1977 .

[2]  Márcio José Coelho Pontes,et al.  Classification of extra virgin olive oil and verification of adulteration using digital images and discriminant analysis , 2015 .

[3]  Sergey Kucheryavskiy,et al.  Determination of fat and total protein content in milk using conventional digital imaging. , 2014, Talanta.

[4]  Sajad Kiani,et al.  Fusion of artificial senses as a robust approach to food quality assessment , 2016 .

[5]  Andrey Bogomolov,et al.  Selecting optimal wavelength intervals for an optical sensor: A case study of milk fat and total protein analysis in the region 400-1100 nm , 2015 .

[6]  Xin Sun,et al.  Prediction of pork loin quality using online computer vision system and artificial intelligence model. , 2018, Meat science.

[7]  José Blasco,et al.  Application for the estimation of the standard citrus colour index (CCI) using image processing in mobile devices , 2018 .

[8]  I. K. Sawhney,et al.  Algorithm for automatic calibration of color vision system in foods , 2018, Journal of Food Measurement and Characterization.

[9]  Márcio José Coelho Pontes,et al.  Classification of edible vegetable oil using digital image and pattern recognition techniques , 2014 .

[10]  Anil K. Jain,et al.  39 Dimensionality and sample size considerations in pattern recognition practice , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[11]  K. Baumann,et al.  Chemoinformatic Classification Methods and their Applicability Domain , 2016, Molecular informatics.

[12]  M. Barker,et al.  Partial least squares for discrimination , 2003 .

[13]  Hui Wang,et al.  Collaborative representation based classifier with partial least squares regression for the classification of spectral data , 2018, Chemometrics and Intelligent Laboratory Systems.

[14]  P. Cano Marchal,et al.  Expert system based on computer vision to estimate the content of impurities in olive oil samples , 2013 .

[15]  Manabu Kano,et al.  Sparse Sample Regression Based Just-In-Time Modeling (SSR-JIT): Beyond Locally Weighted Approach , 2016 .

[16]  A. Bogomolov,et al.  Scatter-based quantitative spectroscopic analysis of milk fat and total protein in the region 400-1100nm in the presence of fat globule size variability , 2013 .

[17]  Xiuqin Rao,et al.  Computer vision detection of surface defect on oranges by means of a sliding comparison window local segmentation algorithm , 2017, Comput. Electron. Agric..

[18]  Roberto Quevedo,et al.  Evaluation of Food Quality and Safety with Hyperspectral Imaging (HSI) , 2016, Food Engineering Reviews.

[19]  Megha P. Arakeri,et al.  Computer Vision Based Fruit Grading System for Quality Evaluation of Tomato in Agriculture industry , 2016 .

[20]  María Jesús Lerma-García,et al.  Authentication of extra virgin olive oils by Fourier-transform infrared spectroscopy , 2010 .

[21]  Lorenzo Cerretani,et al.  Chemometric applications to assess quality and critical parameters of virgin and extra-virgin olive oil. A review. , 2016, Analytica chimica acta.

[22]  Stilianos Siskos,et al.  An approach for estimating adulteration of virgin olive oil with soybean oil using image analysis , 2017, 2017 6th International Conference on Modern Circuits and Systems Technologies (MOCAST).

[23]  Satyam Srivastava,et al.  Quality assessment of commercial bread samples based on breadcrumb features and freshness analysis using an ultrasonic machine vision (UVS) system , 2015, Journal of Food Measurement and Characterization.

[24]  Hoeil Chung,et al.  Use of temperature dependent Raman spectra to improve accuracy for analysis of complex oil-based samples: lube base oils and adulterated olive oils. , 2012, Analytica Chimica Acta.

[25]  Vladislav Galyanin,et al.  Building global models for fat and total protein content in raw milk based on historical spectroscopic data in the visible and short-wave near infrared range. , 2016, Food chemistry.

[26]  Vincenzo Chiofalo,et al.  Fusion of electronic nose, electronic tongue and computer vision for animal source food authentication and quality assessment – A review , 2017 .

[27]  Franco Pedreschi,et al.  Statistical pattern recognition classification with computer vision images for assessing the furan content of fried dough pieces. , 2018, Food chemistry.

[28]  Antonietta Baiano,et al.  Applications of hyperspectral imaging for quality assessment of liquid based and semi-liquid food products: A review , 2017 .

[29]  Saeid Minaei,et al.  Honey characterization using computer vision system and artificial neural networks. , 2014, Food chemistry.

[30]  M. de la Guardia,et al.  Smartphone determination of fat in cured meat products , 2017 .

[31]  Dong-Sheng Cao,et al.  An overview of variable selection methods in multivariate analysis of near-infrared spectra , 2019, TrAC Trends in Analytical Chemistry.

[32]  Manabu Kano,et al.  Estimation of active pharmaceutical ingredients content using locally weighted partial least squares and statistical wavelength selection. , 2011, International journal of pharmaceutics.

[33]  Manabu Kano,et al.  Locally weighted kernel partial least squares regression based on sparse nonlinear features for virtual sensing of nonlinear time-varying processes , 2017, Comput. Chem. Eng..

[34]  J-H Liu,et al.  Predicting pork loin intramuscular fat using computer vision system. , 2018, Meat science.

[35]  Valeria Belikova,et al.  Reference-free spectroscopic determination of fat and protein in milk in the visible and near infrared region below 1000nm using spatially resolved diffuse reflectance fiber probe. , 2017, Talanta.

[36]  Di Wu,et al.  Colour measurements by computer vision for food quality control – A review , 2013 .