Function from visual analysis and physical interaction: a methodology for recognition of generic classes of objects

The GRUFF-I (Generic Recognition Using Form, Function and Interaction) system reasons about and generates plans for interaction with 3-D shapes for the purpose of generic object recognition. A researcher selects an object and places it in an observation area. An initial intensity and range image are acquired and provided as input to a three-stage recognition system. The first stage builds a 3-D model. The second stage considers the shape-suggested functionality of this model by applying concepts of physics and causation (e.g., to infer stability) to label the object's potential functionality. The third stage uses this labeling to instantiate a plan for interaction to confirm the object's functional use in a task by incorporating feedback from both visual and robotic sensors. Results of this work are presented for eighteen chair-like and cup-like objects. Major conclusions from this work include: (1) metrically accurate representations of the world can be built and used for higher level reasoning, (2) shape-based reasoning prior to interaction-based reasoning provides an efficient methodology for object recognition, in terms of the judicious use of system resources, and (3) interaction-based reasoning helps to confirm the functionality of a categorized object without explicitly determining the object's material composition.