Acting and Interacting in the Real World

A characteristic of robots that distinguishes them from other intelligent agents like the chess playing Deep Blue [1] or any 2D image classifier [2] is its physical embodiment in the real world. A robot can act in or interact with the environment and thereby change it. To do this in any meaningful way, it needs to be able to perceive and represent the 3D structure of its surrounding. This facilitates processes like motion planning of actuators or grasping and manipulation. Recently, through the release of devices like the kinect [3], sensing of dense and high quality 3D data became cheap, fast and easy. Certain constraints introduced by other 3D sensing devices are removed. However, the basic challenges of processing and understanding 3D data remain. These are (i) the high dimensionality of the data, (ii) noise, (iii) occlusions and most importantly (iv) how to get from the low-level set of 3D points to high-level semantic information. In the following, we will outline how we deal with these challenges given the specific task of grasping and manipulation in a table top scenario.