Learning the Geometric Meaning of Symbolic Abstractions for Manipulation Planning

We present an approach for learning a mapping between geometric states and logical predicates. This mapping is a necessary part of any robotic system that requires task-level reasoning and path planning. Consider a robot tasked with putting a number of cups on a tray. To achieve the goal the robot needs to find positions for all the objects, and if necessary may need to stack one cup inside another to get them all on the tray. This requires translating back and forth between symbolic states that the planner uses such as “stacked(cup1,cup2)” and geometric states representing the positions and poses of the objects. The mapping we learn in this paper achieves this translation. We learn it from labelled examples, and significantly, learn a representation that can be used in both the forward (from geometric to symbolic) and reverse directions. This enables us to build symbolic representations of scenes the robot observes, and also to translate a desired symbolic state from a plan into a geometric state that the robot can actually achieve through manipulation. We also show how the approach can be used to generate significantly different geometric solutions to support backtracking. We evaluate the work both in simulation and on a robot arm.