Development of a Mobile Robotic Phenotyping System for Growth Chamber-based Studies of Genotype x Environment Interactions

Abstract: To increase understanding of the interaction between phenotype and genotype x environment to improve crop performance, large amounts of phenotypic data are needed. Studying plants of a given strain under multiple environments can greatly help to reveal their interactions. To collect the labor-intensive data required to perform experiments in this area, a Mecanum-wheeled, magnetic-tape-following indoor rover has been developed to accurately and autonomously move between and inside growth chambers. Integration of the motor controllers, a robot arm, and a Microsoft Kinect (v2) 3D sensor was achieved in a customized C++ program. Detecting and segmenting plants in a multi-plant environment is a challenging task, which can be aided by integration of depth data into these algorithms. Image-processing functions were implemented to filter the depth image to minimize noise and remove undesired surfaces, reducing the memory requirement and allowing the plant to be reconstructed at a higher resolution in real-time. Three-dimensional meshes representing plants inside the chamber were reconstructed using the Kinect SDK’s KinectFusion. After transforming user-selected points in camera coordinates to robot-arm coordinates, the robot arm is used in conjunction with the rover to probe desired leaves, simulating the future use of sensors such as a fluorimeter and Raman spectrometer. This paper reports the system architecture and some preliminary results of the system.

[1]  B. Faverjon,et al.  Probabilistic Roadmaps for Path Planning in High-Dimensional Con(cid:12)guration Spaces , 1996 .

[2]  Matthew N. Dailey,et al.  Automatic morphological trait characterization for corn plants via 3D holographic reconstruction , 2014 .

[3]  Jack Bresenham,et al.  Algorithm for computer control of a digital plotter , 1965, IBM Syst. J..

[4]  Torsten Kröger.  Online Trajectory Generation: Straight-Line Trajectories , 2011, IEEE Transactions on Robotics.

[5]  Marina Indri,et al.  A real time distributed approach to collision avoidance for industrial manipulators , 2014, Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA).

[6]  H. Poorter,et al.  Phenotyping plants: genes, phenes and machines. , 2012, Functional plant biology : FPB.

[7]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[8]  Alessandro De Luca,et al.  Integrated control for pHRI: Collision avoidance, detection, reaction and collaboration , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[9]  Hanno Scharr,et al.  A stereo imaging system for measuring plant canopies , 2007 .

[10]  S. LaValle.  Rapidly-exploring random trees : a new tool for path planning , 1998 .

[11]  Seth Hutchinson,et al.  A Framework for Real-time Path Planning in Changing Environments , 2002, Int. J. Robotics Res..

[12]  Anton Gfrerrer.  Geometry and kinematics of the Mecanum wheel , 2008, Comput. Aided Geom. Des..

[13]  D. Straeten,et al.  Seeing is believing: imaging techniques to monitor plant health. , 2001, Biochimica et biophysica acta.

[14]  Nancy M. Amato,et al.  A general framework for PRM motion planning , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[15]  Oussama Khatib,et al.  A depth space approach to human-robot collision avoidance , 2012, 2012 IEEE International Conference on Robotics and Automation.

[16]  George Azzari,et al.  Rapid Characterization of Vegetation Structure with a Microsoft Kinect Sensor , 2013, Sensors.

[17]  John T. Feddema,et al.  Whole arm obstacle avoidance for teleoperated robots , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[18]  Jochen C Reif,et al.  Novel throughput phenotyping platforms in plant genetic studies. , 2007, Trends in plant science.

[19]  John T. Wen,et al.  Real-time robot motion control with circulatory fields , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[20]  Vladimir J. Lumelsky,et al.  Real-time collision avoidance in teleoperated whole-sensitive robot arm manipulators , 1993, IEEE Trans. Syst. Man Cybern..

[21]  RaphaelBertram,et al.  Correction to "A Formal Basis for the Heuristic Determination of Minimum Cost Paths" , 1972 .

[22]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[23]  Friedrich M. Wahl,et al.  Online Trajectory Generation: Basic Concepts for Instantaneous Reactions to Unforeseen Events , 2010, IEEE Transactions on Robotics.

[24]  H. Scharr,et al.  A stereo imaging system for measuring structural parameters of plant canopies. , 2007, Plant, cell & environment.

[25]  Carme Torras,et al.  Robotized Plant Probing: Leaf Segmentation Utilizing Time-of-Flight Data , 2013, IEEE Robotics & Automation Magazine.

[26]  Vladimir J. Lumelsky,et al.  A sensitive skin system for motion control of robot arm manipulators , 1992, Robotics Auton. Syst..

[27]  D. Soyini Madison.  Seeing is believing , 1993 .

[28]  Enric Cervera,et al.  Sensor covering of a robot arm for collision avoidance , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[29]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[30]  Surya P. N. Singh,et al.  V-REP: A versatile and scalable robot simulation framework , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[31]  Daniel Cremers,et al.  Field phenotyping of grapevine growth using dense stereo reconstruction , 2015, BMC Bioinformatics.

[32]  Hanno Scharr,et al.  Modeling leaf growth of rosette plants using infrared stereo image sequences , 2015, Comput. Electron. Agric..

[33]  O. Brock,et al.  Elastic Strips: A Framework for Motion Generation in Human Environments , 2002, Int. J. Robotics Res..

[34]  Thomas Butkiewicz.  Low-cost coastal mapping using Kinect v2 time-of-flight cameras , 2014, 2014 Oceans - St. John's.

[35]  Alin Albu-Schaffer,et al.  Dynamic Motion Planning for Robots in Partially Unknown Environments , 2011 .

[36]  W. Sutherland,et al.  Reaping the Benefits: Science and the sustainable intensification of global agriculture , 2009 .

[37]  Oussama Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Autonomous Robot Vehicles.

[38]  Clifford A. Shaffer,et al.  Real-time robot arm collision detection for telerobotics , 1991 .

[39]  Lydia E. Kavraki,et al.  Analysis of probabilistic roadmaps for path planning , 1998, IEEE Trans. Robotics Autom..

[40]  Alexander Verl,et al.  Real-time path planning for a robot arm in changing environments , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[41]  Alexander H. Waibel,et al.  Skin-Color Modeling and Adaptation , 1998, ACCV.

[42]  Steven M. LaValle,et al.  RRT-connect: An efficient approach to single-query path planning , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[43]  M. G. Rodd.  Introduction to robotics: Mechanics and control: John J. Craig , 1987, Autom..

[44]  Nelson L. Max,et al.  Structured Light-Based 3D Reconstruction System for Plants , 2015, Sensors.

[45]  Roger Y. Tsai,et al.  A new technique for fully autonomous and efficient 3D robotics hand/eye calibration , 1988, IEEE Trans. Robotics Autom..