Model based recognition of specular objects using sensor models

Abstract Recognizing shiny objects with specular reflections is a hard problem for computer vision. Specular reflections appear, disappear, or change their shapes abruptly, due to tiny movements of the viewer. Traditionally, such specular reflections are discarded as annoying noise for recognition purposes. This paper will actively use such specular reflections for recognition. Specular reflections provide distinct clues for object recognition, if properly used. Some advanced sensors, such as underwater sonar or SAR sensors, provide images due to only specular reflections of emitted signals. It is important to establish a technique to recognize objects from such specular images. Although specular reflections are quite unstable, simulating object appearances by using a physics based specular reflection model on top of a geometric modeler allows us to predict specular reflections quite accurately. Recently, several robust matching techniques such as deformable template matchings have been developed. We will employ such physics-based simulator and deformable template matching techniques for specular object recognition. Our system follows the precompilation method. From the specular appearances of an object, the system will extract specular features, collections of pixels arising from specular reflections, and generate a set of (specular) aspects of the object. Specular features comprise stable distinct ones as well as unstable useless ones. The system analyzes the detectability and stability of each specular feature and determines a set of effective specular features to be used for specular aspect classification. For each specular aspect, the system prepares deformable matching templates. At runtime, an input image is first classified into a few candidate aspects using the predetermined effective features. Some of the specular features are still unstable and misclassification of aspects might occur if we used the binary decision for aspect classification. We will employ a continuous decision making for classification based on the Dempster-Shafer theory instead. Then the system will find the deformable template which provides the best match to verify the existence of the object. In order to demonstrate the usefulness of our system for specular object recognition, we present two experimental results using two different sensors: a TV image of a shiny object and a synthetic aperture radar (SAR) image of an airplane. The results show the flexibility of the proposed model-based approach for specular object recognition.