Computer vision system for measuring individual cow feed intake using RGB-D camera and deep learning algorithms

Abstract This study entailed the design and implementation of a computer vision system for cow individual feed intake measurement, based on deep Convolutional Neural Networks (CNNs) models, and a low-cost RGB-D (Red, Green, Blue, Depth) camera. Individual feed intake of dairy cows is an important variable currently unavailable in commercial dairies. An RGB-D camera was positioned above the feeding area in an open cowshed. Feed intake was estimated by combining information from the RGB and depth images. Cow identification was conducted using the RGB image. Deep learning algorithms for identification and intake estimation were developed using CNN models. Data for CNN training were acquired by a specially developed automatic data acquisition system. A range of feed weights under varied configurations were collected over a period of seven days with the setup, which included an automatic scale, cameras, and a micro-controller. Test data for feed intake was acquired in an open cowshed research dairy farm, wherein the cows were fed Total Mix Ration (TMR). Images of cows eating over a period of 36 h provided the test data for cow identification. The system was able to accurately identify 93.65% of the cows. The amount of feed consumed, which ranged from 0 to 8 kg per meal, was measured with mean absolute and square errors (MAE and MSE) of 0.127 kg, and 0.034 k g 2 respectively. The analysis showed that the amount and diversity of data are important for model training. Better results were achieved for the model that was trained with high-diversity data than the model trained with homogeneous data (MAE of 1.025 kg, and MSE of 2.845 k g 2 for a model trained on shadow conditions only). Additionally, the training analysis shows that the model based on RGB-D data shows better results than the model based on depth channel data without RGB (MAE of 0.241 kg, and MSE of 0.106 k g 2 ). These results suggest the potential of low-cost cameras for individual feed intake measurements in advanced dairy farms.