The role of feature visibility constraints in perspective alignment

Perspective alignment is a novel method of solving backprojection, the well-known problem of computing three dimensional (3D) position and orientation (pose) of a model from two-dimensional (2D) image features. This paper demonstrates that previous backprojection methods can violate the visibility constraint by computing solution poses in which the model occludes features which should be visible. By definition, these visibility errors are associated with incorrect pose solutions. Yet they occur frequently when previous backprojection methods are used in underconstrained situations. We empirically analyze the frequency and consequences of visibility errors in previous backprojection methods. We then show how perspective alignment satisfies the visibility constraint during the pose solution process to eliminate these errors. The algorithm has been implemented and used in a real-time model-based object tracking system. We describe the algorithm and results of tracking real objects in real-time. The algorithm also has implications for reducing the combinatorics of image-model feature pairing in model-based recognition.