Depth Density Achieves a Better Result for Semantic Segmentation with the Kinect System

Image segmentation is one of the most important methods for animal phenome research. Since the advent of deep learning, many researchers have looked at multilayer convolutional neural networks to solve the problems of image segmentation. A network simplifies the task of image segmentation with automatic feature extraction. Many networks struggle to output accurate details when dealing with pixel-level segmentation. In this paper, we propose a new concept: Depth density. Based on a depth image, produced by a Kinect system, we design a new function to calculate the depth density value of each pixel and bring this value back to the result of semantic segmentation for improving the accuracy. In the experiment, we choose Simmental cattle as the target of image segmentation and fully convolutional networks (FCN) as the verification networks. We proved that depth density can improve four metrics of semantic segmentation (pixel accuracy, mean accuracy, mean intersection over union, and frequency weight intersection over union) by 2.9%, 0.3%, 11.4%, and 5.02%, respectively. The result shows that depth information produced by Kinect can improve the accuracy of the semantic segmentation of FCN. This provides a new way of analyzing the phenotype information of animals.

[1]  William A Fera.  The next IT challenge. , 2010, Journal of AHIMA.

[2]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Guoping Qiu,et al.  Integrating low-level and semantic features for object consistent segmentation , 2013, Neurocomputing.

[4]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  W. Johannsen,et al.  The Genotype Conception of Heredity , 1911, The American Naturalist.

[6]  Urs Schmidhalter,et al.  Evaluating RGB Imaging and Multispectral Active and Hyperspectral Passive Sensing for Assessing Early Plant Vigor in Winter Wheat , 2018, Sensors.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  H. R. Siebner,et al.  From the genome to the phenome and back: linking genes with human brain function and structure using genetically informed neuroimaging , 2009, Neuroscience.

[10]  Gensheng Hu,et al.  Self-adversarial Training and Attention for Multi-task Wheat Phenotyping , 2019 .

[11]  Ming-Jer Tsai,et al.  Identifying Fagaceae Species in Taiwan Using Leaf Images , 2019 .

[12]  Malia A. Gehan,et al.  Lights, camera, action: high-throughput plant phenotyping is ready for a close-up. , 2015, Current opinion in plant biology.

[13]  Dazhe Zhao,et al.  Image segmentation and bias correction using local inhomogeneous iNtensity clustering (LINC): A region-based level set method , 2017, Neurocomputing.

[14]  Huijie Zhao,et al.  A High Throughput Integrated Hyperspectral Imaging and 3D Measurement System , 2018, Sensors.

[15]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Tao Cheng,et al.  Combining computer vision and deep learning to enable ultra-scale aerial phenotyping and precision agriculture: A case study of lettuce production , 2019, Horticulture Research.

[17]  ZhaoDazhe,et al.  Image segmentation and bias correction using local inhomogeneous iNtensity clustering (LINC) , 2017 .

[18]  Thomas Blaschke,et al.  Evaluation of Different Machine Learning Methods and Deep-Learning Convolutional Neural Networks for Landslide Detection , 2019, Remote. Sens..

[19]  SchmidhuberJürgen.  Deep learning in neural networks , 2015 .

[20]  Tao Chang,et al.  Scene-Aware Deep Networks for Semantic Segmentation of Images , 2019, IEEE Access.

[21]  Lie Tang,et al.  Automated morphological traits extraction for sorghum plants via 3D point cloud data analysis , 2019, Comput. Electron. Agric..

[22]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[23]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[24]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[25]  Shih-Fu Chang,et al.  CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  N. Schork,et al.  Genetics of complex disease: approaches, problems, and solutions. , 1997, American journal of respiratory and critical care medicine.

[27]  Subhransu Maji,et al.  Bilinear CNN Models for Fine-Grained Visual Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[28]  Tyrone D. Cannon,et al.  Phenomics: the systematic study of phenotypes on a genome-wide scale , 2009, Neuroscience.

[29]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[30]  Ping Wang,et al.  Image compression with meanshift based inverse colorization , 2013, 2013 IEEE International Conference on Consumer Electronics (ICCE).

[31]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[32]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[33]  S. Omholt,et al.  Phenomics: the next challenge , 2010, Nature Reviews Genetics.

[34]  M. Tester,et al.  Phenomics--technologies to relieve the phenotyping bottleneck. , 2011, Trends in plant science.

[35]  Cyrill Stachniss,et al.  REAL-TIME BLOB-WISE SUGAR BEETS VS WEEDS CLASSIFICATION FOR MONITORING FIELDS USING CONVOLUTIONAL NEURAL NETWORKS , 2017 .

[36]  Jin Tae Kwak,et al.  Deep convolutional neural network for classifying Fusarium wilt of radish from unmanned aerial vehicles , 2017 .

[37]  Yuan Yuan,et al.  Contour-aware network for semantic segmentation via adaptive depth , 2018, Neurocomputing.

[38]  Farid Melgani,et al.  Computer vision-based phenotyping for improvement of plant productivity: a machine learning perspective , 2018, GigaScience.

[39]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Ulrich Schurr,et al.  Future scenarios for plant phenotyping. , 2013, Annual review of plant biology.

[41]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[42]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[43]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Meng Liu,et al.  Three-Dimensional Reconstruction of Soybean Canopies Using Multisource Imaging for Phenotyping Analysis , 2018, Remote. Sens..

[45]  Jeffrey W. White,et al.  Field-based phenomics for plant genetics research , 2012 .

[46]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Jian Sun,et al.  Accelerating Very Deep Convolutional Networks for Classification and Detection , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.