Grasping and Object Reorientation for Autonomous Construction of Stone Structures

Building large and stable structures from highly irregular stones is among the most challenging construction tasks with excavators. In this letter, we present a method for grasp planning and object manipulation that enables the world's first autonomous assembly of a large-scale stone wall with an unmanned hydraulic excavator system. Our method utilizes point clouds and mesh surface reconstruction of stones in order to plan grasp configurations with a 2-jaw gripper mounted on the excavator. Besides considering the geometry of the stone to sample force closure grasps, the grasp planner also takes into account collision constraints during object pick and place, informed by a LiDAR based point cloud map. Furthermore, we show an approach to reorient arbitrarily shaped objects that are not feasible to be directly placed at the desired location without violating collision constraints. Using a physics engine, we find a settled intermediate pose that allows direct placement and is reachable from the initial stone pose. The applicability of the proposed grasp planning method is demonstrated with the construction of a dry stone wall composed of over one hundred boulders using an autonomous excavator. We show a high primary grasp success rate (82.2 $\%$) and illustrate how the system recovers from slippage by relocating the object and re-planning the grasp correspondingly.