Manhattan World: Orientation and Outlier Detection by Bayesian Inference

This letter argues that many visual scenes are based on a Manhattan three-dimensional grid that imposes regularities on the image statistics. We construct a Bayesian model that implements this assumption and estimates the viewer orientation relative to the Manhattan grid. For many images, these estimates are good approximations to the viewer orientation (as estimated manually by the authors). These estimates also make it easy to detect outlier structures that are unaligned to the grid. To determine the applicability of the Manhattan world model, we implement a null hypothesis model that assumes that the image statistics are independent of any three-dimensional scene structure. We then use the log-likelihood ratio test to determine whether an image satisfies the Manhattan world assumption. Our results show that if an image is estimated to be Manhattan, then the Bayesian model's estimates of viewer direction are almost always accurate (according to our manual estimates), and vice versa.

[1]  J. Powell.  Mathematical Methods in Physics , 1965 .

[2]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[3]  Yoshiaki Shirai,et al.  Three-Dimensional Computer Vision , 1987, Symbolic Computation.

[4]  Yiannis Aloimonos,et al.  Active vision , 2004, International Journal of Computer Vision.

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  Beatrice Brillault-O'Mahony,et al.  New method for vanishing point detection , 1991, CVGIP Image Underst..

[7]  Andrew Zisserman,et al.  Geometric invariance in computer vision , 1992 .

[8]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[9]  Jaime López-Krahe,et al.  Contribution to the Determination of Vanishing Points Using Hough Transform , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  B. D. Guenther,et al.  Aided and automatic target recognition based upon sensory inputs from image forming systems , 1997 .

[11]  Song Chun ZhuDept.  Clutter Modeling and Performance Analysis in Automatic Target Recognition 1 , 1998 .

[12]  Andrew Zisserman,et al.  Robust computation and parametrization of multiple view relations , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[13]  Jefferey A. Shufelt,et al.  Performance Evaluation and Analysis of Vanishing Point Detection Techniques , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Alan L. Yuille,et al.  Fundamental bounds on edge detection: an information theoretic evaluation of different edge cues , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[15]  David Mumford,et al.  Statistics of natural images and models , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[16]  Alan L. Yuille,et al.  Manhattan World: compass direction from a single image by Bayesian inference , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[17]  Alan L. Yuille,et al.  Order parameters for minimax entropy distributions: when does high level knowledge help? , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[18]  Norberto M. Grzywacz,et al.  The Minimal Local-Asperity Hypothesis of Early Retinal Lateral Inhibition , 2000, Neural Computation.

[19]  Alan L. Yuille,et al.  The Manhattan World Assumption: Regularities in Scene Statistics which Enable Bayesian Inference , 2000, NIPS.

[20]  Alan L. Yuille,et al.  Fundamental Limits of Bayesian Inference: Order Parameters and Phase Transitions for Road Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  David W. Jacobs,et al.  In search of illumination invariants , 2001, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[22]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[23]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[24]  Michael Isard,et al.  BraMBLe: a Bayesian multiple-blob tracker , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[25]  M. Isard,et al.  Automatic Camera Calibration from a Single Manhattan Image , 2002, ECCV.

[26]  Alan L. Yuille,et al.  Statistical Edge Detection: Learning and Evaluating Edge Cues , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Alan L. Yuille,et al.  Order Parameters for Detecting Target Curves in Images: When Does High Level Knowledge Help? , 2004, International Journal of Computer Vision.

[28]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.