Exploration of Autonomous Mobile Robots through Challenging Outdoor Environments for Natural Plant Recognition Using Deep Neural Network

Modernization of living environments and human activities have severe effects on many parameters and factors such as climate change and global warming, an increase of incidence and the severity of wildfires, land surfaces and ice sheets, ecological imbalance, change of fertility of the soil, flow of energy, food security, etc. In addition, human modernization has had a negative impact on biodiversity and the natural environment. An integral component of modernization is agriculture that associates with the outdoor environment and relevant issues. Automation of agricultural activities contributes to reducing the dependency on human labor and the harmful effects on the natural environment. The correct identification of plants in outdoor environments has been neglected and many critical environmental and non-environmental limits and factors, such as weather conditions, time, viewpoint, lighting conditions (illuminations and light intensity), distance, etc., have not been considered in existing plant recognition systems and technologies. Hence, there is a demand to develop mobile and real-time systems for plant recognition in natural environments. This paper addresses these challenges and introduces the application of autonomous mobile robot and semi-robots for recognition of natural plant species in outdoor environments. Furthermore, the proposed system presents the use of employing low-cost cameras, such as iPhone 6s, Canon EOS 600D and Samsung Galaxy Note 4, for plant recognition system in real-time. The performance of the system has been tested with a number of experiments in different years, 2017 and 2018, and at different times of day, morning and evening. The proposed system is a combination of new technologies involving deep learning concepts and an autonomous field robot to carry out precise plant recognition in challenging environments. The final accuracy of the mobile real-time system is 84.1666%.

[1]  Klaus-Dieter Kuhnert,et al.  Modern Detection and Description Methods for Natural Plants Recognition , 2017 .

[2]  Klaus-Dieter Kuhnert,et al.  Fully-automatic natural plant recognition system using deep neural network for dynamic outdoor environments , 2019, SN Applied Sciences.

[3]  Gulam Amer,et al.  Design and operation of Wi-Fi agribot integrated system , 2015, 2015 International Conference on Industrial Instrumentation and Control (ICIC).

[4]  W. R. Sam Emmanuel,et al.  Texture Feature Extraction for Identification of Medicinal Plants and comparison of different classifiers , 2013 .

[5]  Uda Hashim,et al.  Embedded portable device for herb leaves recognition using image processing techniques and neural network algorithm , 2012 .

[6]  Megha P. Arakeri,et al.  Computer vision based robotic weed control system for precision agriculture , 2017, 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[7]  N. Zhang,et al.  Precision agriculture—a worldwide overview , 2002 .

[8]  Heping Zhu,et al.  Robust Crop and Weed Segmentation under Uncontrolled Outdoor Illumination , 2011, Sensors.

[9]  Klaus-Dieter Kuhnert,et al.  Efficient Modern Description Methods by Using SURF Algorithm for Recognition of Plant Species , 2015 .

[10]  Ji Li,et al.  Crop recognition under weedy conditions based on 3 D imaging for robotic weed control , 2019 .

[11]  Thomas R. Kurfess,et al.  Robotics And Automation Handbook , 2019 .

[12]  Daniele Nardi,et al.  Fast and Accurate Crop and Weed Identification with Summarized Train Sets for Precision Agriculture , 2016, IAS.

[13]  Lie Tang,et al.  Crop recognition under weedy conditions based on 3D imaging for robotic weed control , 2018, J. Field Robotics.

[14]  Li WangDong-Chen He,et al.  Texture classification using texture spectrum , 1990, Pattern Recognit..

[15]  I. Gogul,et al.  Smart Autonomous Gardening Rover with Plant Recognition Using Neural Networks , 2016 .

[16]  Faruq Mohammad,et al.  Feature decision-making ant colony optimization system for an automated recognition of plant species , 2015, Expert Syst. Appl..

[17]  Claus G. Sørensen,et al.  HortiBot: A System Design of a Robotic Tool Carrier for High-tech Plant Nursing , 2007 .

[18]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[19]  Ji Li.  3D machine vision system for robotic weeding and plant phenotyping , 2014 .

[20]  Klaus-Dieter Kuhnert,et al.  Automatic plant recognition system for challenging natural plant species , 2017 .

[21]  Aiswarya Mohan,et al.  Automatic Weed Detection System And Smart Herbicide Sprayer Robot For Corn Fields , 2016 .

[22]  Joseph F. Murray,et al.  Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation , 2010, Neural Computation.

[23]  Dong-Chen He,et al.  Texture Unit, Texture Spectrum, And Texture Analysis , 1990 .

[24]  Lauren Reiter,et al.  Zephyr , 2013 .

[25]  Albert-Jan Baerveldt,et al.  An Agricultural Mobile Robot with Vision-Based Perception for Mechanical Weed Control , 2002, Auton. Robots.

[26]  Klaus-Dieter Kuhnert,et al.  COMPARISON OF MODERN DESCRIPTION METHODS FOR THE RECOGNITION OF 32 PLANT SPECIES , 2015 .

[27]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[28]  Hod Lipson,et al.  Understanding Neural Networks Through Deep Visualization , 2015, ArXiv.

[29]  Berrin A. Yanikoglu,et al.  Automatic plant identification from photographs , 2014, Machine Vision and Applications.

[30]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.