The virtual lands of Oz: testing an agribot in simulation

Testing autonomous robots typically requires expensive test campaigns in the field. To alleviate them, a promising approach is to perform intensive tests in virtual environments. This paper presents an industrial case study on the feasibility and effectiveness of such an approach. The subject system is Oz, an agriculture robot for autonomous weeding. Its software was tested with weeding missions in virtual crop fields, using a 3D simulator based on Gazebo. The case study faced several challenges: the randomized generation of complex 3D environments, the automated checking of the robot behavior (test oracle), and the imperfect fidelity of simulation with respect to real-world behavior. We describe the test approach we developed, and compare the results with the ones of the industrial field tests. Despite the low-fidelity physics of the robot, the virtual tests revealed most software issues found in the field, including a major one that caused the majority of failures; they also revealed a new issue missed in the field. On the downside, the simulation could introduce spurious failures that would not occur in the real world.