Unsupervised deep learning and semi-automatic data labeling in weed discrimination

Abstract In recent years, supervised Deep Neural Networks have achieved the state-of-the-art in image recognition and this success has spread in many areas. In agricultural field, several researches have been conducted using architectures such as Convolutional Neural Networks. Despite this success, these works are still highly dependent on very time–costly manual data labeling. In contrast to this scenario, Unsupervised Deep Learning has no dependency on data labeling and is targeted as the future of the area, but after a promising start has been obfuscated by the success of supervised networks. Meanwhile, the low-cost of acquisition of field crop imagery using Unnamed Aerial Vehicles could be largely boosted in real-world applications if these images could be annotated without the need for a human specialist. In this work, we tested two recent unsupervised deep clustering algorithms, Joint Unsupervised Learning of Deep Representations and Image Clusters (JULE) and Deep Clustering for Unsupervised Learning of Visual Features (DeepCluster), using two public weed datasets. The first dataset was captured in a soybean plantation in Brazil and discriminates weeds between grass and broadleaf. The second dataset consists of 17,509 labeled images of eight nationally significant weed species native to Australia. We evaluated the purely unsupervised clustering performance using the NMI and Unsupervised Clustering Accuracy metrics and analysed the effects of techniques like data augmentation and transfer learning to improve clustering quality in a broad discussion that can be useful for unsupervised deep clustering in general. We also propose the usage of semi-automatic data labeling which greatly reduces the cost of manual data labeling and can be easily replicated to different datasets. This approach achieved 97% accuracy in discrimination of grass and broadleaf while reducing the number of manual annotations by 100 times, using a custom set of training images, without images labeled using inaccurate clusters.

[1]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Abdolabbas Jafari,et al.  Evaluation of support vector machine and artificial neural networks in weed detection using shape features , 2018, Comput. Electron. Agric..

[3]  Andreas Kamilaris,et al.  Deep learning in agriculture: A survey , 2018, Comput. Electron. Agric..

[4]  Daniele Nardi,et al.  Fast and Accurate Crop and Weed Identification with Summarized Train Sets for Precision Agriculture , 2016, IAS.

[5]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Wen Zhang,et al.  A review on weed detection using ground-based machine vision and image processing techniques , 2019, Comput. Electron. Agric..

[7]  Yang Xu,et al.  Weed identification based on K-means feature learning combined with convolutional neural network , 2017, Comput. Electron. Agric..

[8]  Adel Hafiane,et al.  Deep Learning with Unsupervised Data Labeling for Weed Detection in Line Crops in UAV Images , 2018, Remote. Sens..

[9]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[10]  Mostafa Rahimi Azghadi,et al.  DeepWeeds: A Multiclass Weed Species Image Dataset for Deep Learning , 2018, Scientific Reports.

[11]  Jizhong Deng,et al.  A fully convolutional network for weed mapping of unmanned aerial vehicle (UAV) imagery , 2018, PloS one.

[12]  Rasmus Nyholm Jørgensen,et al.  RoboWeedSupport - Detection of weed locations in leaf occluded cereal crops using a fully convolutional neural network , 2017 .

[13]  Arnold W. Schumann,et al.  Deep learning for image-based weed detection in turfgrass , 2019, European Journal of Agronomy.

[14]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[15]  Aydin Kaya,et al.  Analysis of transfer learning for deep neural network based plant classification models , 2019, Comput. Electron. Agric..

[16]  Dhruv Batra,et al.  Joint Unsupervised Learning of Deep Representations and Image Clusters , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[18]  Hemerson Pistori,et al.  Weed detection in soybean crops using ConvNets , 2017, Comput. Electron. Agric..

[19]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[20]  Cai Cheng,et al.  Weed seeds classification based on PCANet deep learning baseline , 2015, 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[21]  Yi Yang,et al.  Image Clustering Using Local Discriminant Models and Global Integration , 2010, IEEE Transactions on Image Processing.

[22]  Hubert Cecotti,et al.  Semi-automatic ground truth generation using unsupervised clustering and limited manual labeling: Application to handwritten character recognition , 2015, Pattern Recognit. Lett..

[23]  Roland Siegwart,et al.  weedNet: Dense Semantic Weed Classification Using Multispectral Images and MAV for Smart Farming , 2017, IEEE Robotics and Automation Letters.

[24]  Cyrill Stachniss,et al.  UAV-based crop and weed classification for smart farming , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).