Object tracking by comparing multiple viewpoint images to CG images

There has recently been much research into real-time tracking of object location and orientation tracking. However, many of the current methods use explicit image features such as corners, edges or silhouettes, etc. and are, therefore, not well suited to tracking curved and smooth objects from which it is difficult to extract features. On the other hand, with methods that attempt to estimate the movement of an object from the optical flow obtained from the pixel intensity gradient, it is difficult to avoid tracking errors accumulated during tracking. Therefore, in this paper we propose a method for estimating the movement of an object with six degrees of freedom without extracting features from the images. In particular, the difference between real images and CG (computer graphics) images is minimized based on the principle of intensity gradient images. CG images can be generated using an object model which consists of shape and color information with location and orientation parameters of the object. Here we show that by using images obtained simultaneously from multiple cameras at different angles, the stability of this tracking method is improved. We present experiments in which we construct an object tracking system that uses multiple cameras and show that we can perform object tracking with six degrees of freedom at a speed of approximately five frames per second for a rigid and arbitrarily curved object from which it is difficult to extract features. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(13): 28–39, 2006; Published online in Wiley InterScience (). DOI 10.1002sscj.20546