Automatic Monitoring Cheese Ripeness Using Computer Vision and Artificial Intelligence

Ripening is a very important process that contributes to cheese quality, as its characteristics are determined by the biochemical changes that occur during this period. Therefore, monitoring ripening time is a fundamental task to market a quality product in a timely manner. However, it is difficult to accurately determine the degree of cheese ripeness. Although some scientific methods have also been proposed in the literature, the conventional methods adopted in dairy industries are typically based on visual and weight control. This study proposes a novel approach aimed at automatically monitoring the cheese ripening based on the analysis of cheese images acquired by a photo camera. Both computer vision and machine learning techniques have been used to deal with this task. The study is based on a dataset of 195 images (specifically collected from an Italian dairy industry), which represent Pecorino cheese forms at four degrees of ripeness. All stages but the one labeled as “day 18”, which has 45 images, consist of 50 images. These images have been handled with image processing techniques and then classified according to the degree of ripening, i.e., 18, 22, 24, and 30 days. A 5-fold cross-validation strategy was used to empirically evaluate the performance of the models. During this phase, each training fold was augmented online. This strategy allowed to use 624 images for training, leaving 39 original images per fold for testing. Experimental results have demonstrated the validity of the approach, showing good performance for most of the trained models.

[1]  J. Adamowski,et al.  A deep learning image segmentation model for agricultural irrigation system classification , 2022, Comput. Electron. Agric..

[2]  M. Cord,et al.  Transformer Decoders with MultiModal Regularization for Cross-Modal Food Retrieval , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Kai Chen,et al.  Egeria: Efficient DNN Training with Knowledge-Guided Layer Freezing , 2022, EuroSys.

[4]  Zbigniew Leonowicz,et al.  Dermatologist-Level Classification of Skin Cancer Using Cascaded Ensembling of Convolutional Neural Network and Handcrafted Features Based Deep Neural Network , 2022, IEEE Access.

[5]  C. Pappas,et al.  FT-MIR Analysis of Water-Soluble Extracts during the Ripening of Sheep Milk Cheese with Different Phospholipid Content , 2021, Dairy.

[6]  Yousef Abbaspour-Gilandeh,et al.  Detection of fraud in ginger powder using an automatic sorting system based on image processing technique and deep learning , 2021, Comput. Biol. Medicine.

[7]  Yousef Abbaspour-Gilandeh,et al.  A novel method based on machine vision system and deep learning to detect fraud in turmeric powder , 2021, Comput. Biol. Medicine.

[8]  S. Vimal,et al.  Recognition of food type and calorie estimation using neural network , 2021, J. Supercomput..

[9]  Cecilia Di Ruberto,et al.  Invariant Moments, Textural and Deep Features for Diagnostic MR and CT Image Retrieval , 2021, CAIP.

[10]  Jitendra Virmani,et al.  Hybrid computer-aided classification system design using lightweight end-to-end Pre-trained CNN-based deep feature extraction and PCA-SVM classifier for chest radiographs , 2021, Deep Learning for Chest Radiographs.

[11]  Cecilia Di Ruberto,et al.  A novel deep learning based approach for seed image classification and retrieval , 2021, Comput. Electron. Agric..

[12]  Mohamed Hammami,et al.  Abnormal crowd density estimation in aerial images based on the deep and handcrafted features fusion , 2021, Expert Syst. Appl..

[13]  Giorgio Giacinto,et al.  Convolutional neural networks for relevance feedback in content based image retrieval , 2020, Multimedia Tools and Applications.

[14]  Peilin Zhang,et al.  Gear Fault Feature Extraction Based on Fuzzy Function and Improved Hu Invariant Moments , 2020, IEEE Access.

[15]  Feature Extraction and Image Processing for Computer Vision , 2020 .

[16]  Xiaoxiao Li,et al.  Geometrically Invariant Color Medical Image Null-Watermarking Based on Precise Quaternion Polar Harmonic Fourier Moments , 2019, IEEE Access.

[17]  Da-Wen Sun,et al.  Developments of nondestructive techniques for evaluating quality attributes of cheeses: A review , 2019, Trends in Food Science & Technology.

[18]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[19]  Steven C. H. Hoi,et al.  Learning Cross-Modal Embeddings With Adversarial Networks for Cooking Recipes and Food Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Ana M. Jiménez-Carvelo,et al.  Alternative data mining/machine learning methods for the analytical evaluation of food quality and authenticity - A review. , 2019, Food research international.

[21]  Wen Zhang,et al.  A review on weed detection using ground-based machine vision and image processing techniques , 2019, Comput. Electron. Agric..

[22]  Anne Humeau-Heurtier,et al.  Texture Feature Extraction Methods: A Survey , 2019, IEEE Access.

[23]  Jenq-Neng Hwang,et al.  Multi-Camera Tracking of Vehicles based on Deep Features Re-ID and Trajectory-Based Camera Link Models , 2019, CVPR Workshops.

[24]  Paolo Napoletano,et al.  CNN-based features for retrieval and classification of food images , 2018, Comput. Vis. Image Underst..

[25]  Reinhard Klette,et al.  Detection of Red-Meat Adulteration by Deep Spectral-Spatial Features in Hyperspectral Images , 2018, J. Imaging.

[26]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[27]  Cecilia Di Ruberto,et al.  Fast and accurate computation of orthogonal moments for texture analysis , 2018, Pattern Recognit..

[28]  Kosin Chamnongthai,et al.  The Design and Implementation of an Ingredient-Based Food Calorie Estimation System Using Nutrition Knowledge and Fusion of Brightness and Heat Information , 2018, IEEE Access.

[29]  Cecilia Di Ruberto,et al.  Rotation Invariant Co-occurrence Matrix Features , 2017, ICIAP.

[30]  Paolo Napoletano,et al.  Learning CNN-based Features for Retrieval of Food Images , 2017, ICIAP Workshops.

[31]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[32]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[34]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[37]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Mohammed Fakir,et al.  Image Annotation by Moments , 2014 .

[39]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[40]  Isabel Revilla,et al.  Prediction of the type of milk and degree of ripening in cheeses by means of artificial neural networks with data concerning fatty acids and near infrared spectroscopy. , 2013, Talanta.

[41]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[42]  Ming Yang,et al.  Large-scale image classification: Fast feature extraction and SVM training , 2011, CVPR 2011.

[43]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  G. Corrieu,et al.  Initial studies into the characterisation of ripening stages of Emmental cheeses by mid-infrared spectroscopy , 2009 .

[45]  Derek Bradley,et al.  Adaptive Thresholding using the Integral Image , 2007, J. Graph. Tools.

[46]  C. Giardina,et al.  Application of FT-NIR and FT-IR spectroscopy to study the shelf-life of Crescenza cheese , 2005 .

[47]  P. McSweeney Biochemistry of Cheese Ripening , 2004 .

[48]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[49]  P. Fox,et al.  Acceleration of cheese ripening , 1996, Antonie van Leeuwenhoek.

[50]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Sim Heng Ong,et al.  Image Analysis by Tchebichef Moments , 2001, IEEE Trans. Image Process..

[52]  G. Fitzgerald,et al.  Biotechnological approaches to the understanding and improvement of mature cheese flavour. , 2000, Current opinion in biotechnology.

[53]  M. Devaux,et al.  Phase transition of triglycerides during semi-hard cheese ripening , 2000 .

[54]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[55]  Roland T. Chin,et al.  On image analysis by the methods of moments , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[56]  J. Ross Quinlan,et al.  Learning Efficient Classification Procedures and Their Application to Chess End Games , 1983 .

[57]  M. Teague Image analysis via the general theory of moments , 1980 .

[58]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[59]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.