Estimation of the Horizon in Photographed Outdoor Scenes by Human and Machine

We present three experiments on horizon estimation. In Experiment 1 we verify the human ability to estimate the horizon in static images from only visual input. Estimates are given without time constraints with emphasis on precision. The resulting estimates are used as baseline to evaluate horizon estimates from early visual processes. Stimuli are presented for only ms and then masked to purge visual short-term memory and enforcing estimates to rely on early processes, only. The high agreement between estimates and the lack of a training effect shows that enough information about viewpoint is extracted in the first few hundred milliseconds to make accurate horizon estimation possible. In Experiment 3 we investigate several strategies to estimate the horizon in the computer and compare human with machine “behavior” for different image manipulations and image scene types.

[1]  R. Welch,et al.  Effects of gravitational and optical stimulation on the perception of target elevation , 2001, Perception & psychophysics.

[2]  James M. Coughlan,et al.  Manhattan World: Orientation and Outlier Detection by Bayesian Inference , 2003, Neural Computation.

[3]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[4]  J. Lackner,et al.  Combined influences of gravitoinertial force level and visual field pitch on visually perceived eye level. , 1997, Journal of vestibular research : equilibrium & orientation.

[5]  Mislocalizations of Visual Elevation and Visual Vertical Induced by Visual Pitch: the Great Circle Model a , 1992, Annals of the New York Academy of Sciences.

[6]  Christian Wallraven,et al.  Horizon estimation: perceptual and computational experiments , 2010, APGV '10.

[7]  James J. Gibson,et al.  Determinants of the perceived vertical and horizontal. , 1938 .

[8]  Zijiang J. He,et al.  Perceptual Space in the Dark Affected by the Intrinsic Bias of the Visual System , 2006, Perception.

[9]  H. A. Witkin,et al.  Studies in space orientation; perception of the upright with displaced visual fields. , 1948, Journal of experimental psychology.

[10]  Frank H. Durgin,et al.  Distance Perception and the Visual Horizon in Head-Mounted Displays , 2005, TAP.

[11]  David W. Scott,et al.  Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.

[12]  M. A. Gresty,et al.  Influence of whole-body pitch tilt and kinesthetic cues on the perceived gravity-referenced eye level , 2004, Experimental Brain Research.

[13]  W D Wright,et al.  Color Science, Concepts and Methods. Quantitative Data and Formulas , 1967 .

[14]  Olivier R. Joubert,et al.  How long to get to the “gist” of real-world natural scenes? , 2005 .

[15]  Sarah H Creem-Regehr,et al.  The Importance of a Visual Horizon for Distance Judgments under Severely Degraded Vision , 2011, Perception.

[16]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[17]  Hermann Aubert,et al.  Eine scheinbare bedeutende Drehung von Objecten bei Neigung des Kopfes nach rechts oder links , 1861, Archiv für pathologische Anatomie und Physiologie und für klinische Medicin.

[18]  Zijiang J. He,et al.  Perceiving distance accurately by a directional process of integrating ground information , 2004, Nature.

[19]  M. Isard,et al.  Automatic Camera Calibration from a Single Manhattan Image , 2002, ECCV.

[20]  R. Fernald.  Evolution of eyes , 2000, Current Opinion in Neurobiology.

[21]  Axel Werwatz,et al.  Nonparametric Density Estimation , 2004 .

[22]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[23]  Zijiang J. He,et al.  Distance determined by the angular declination below the horizon , 2001, Nature.

[24]  Margaret A. Hagen,et al.  The Perception of Pictures , 1982 .

[25]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[26]  M M Cohen,et al.  Judgments of eye level in light and in darkness , 1986, Perception & psychophysics.

[27]  J. Gibson.  The Ecological Approach to the Visual Perception of Pictures , 1978 .

[28]  Alexei A. Efros,et al.  Photo clip art , 2007, ACM Trans. Graph..

[29]  J. Philbeck,et al.  Comparison of two indicators of perceived egocentric distance under full-cue and reduced-cue conditions , 1997 .

[30]  Alexei A. Efros,et al.  Putting Objects in Perspective , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[31]  K. F. Austen,et al.  Oculoparalytic Illusion : Visual-Field Dependent Spatial Mislocalizations by Humans Partially Paralyzed with Curare , 2006 .

[32]  Antonio Torralba,et al.  Statistical Context Priming for Object Detection , 2001, ICCV.

[33]  Zijiang J. He,et al.  The linear perspective information in ground surface representation and distance judgment , 2007, Perception & psychophysics.

[34]  Sheena Rogers,et al.  The horizon-ratio relation as information for relative size in pictures , 1996, Perception & psychophysics.

[35]  Michelle R. Greene,et al.  PSYCHOLOGICAL SCIENCE Research Article The Briefest of Glances The Time Course of Natural Scene Understanding , 2022 .

[36]  Alexei A. Efros,et al.  Closing the loop in scene interpretation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Luc Tremblay,et al.  Gender Differences in Perception of Self-Orientation: Software or Hardware? , 2004, Perception.

[38]  Malcolm M. Cohen,et al.  Human spatial orientation in the pitch dimension , 1974 .

[39]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[40]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[41]  Wei Zhang,et al.  Video Compass , 2002, ECCV.

[42]  M. Braunstein,et al.  Background surface and horizon effects in the perception of relative size and distance , 2010, Visual cognition.

[43]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[44]  Tom Foulsham,et al.  Turning the world around: Patterns in saccade direction vary with picture orientation , 2008, Vision Research.

[45]  C Raphel,et al.  Effects of gymnastics expertise on the perception of body orientation in the pitch dimension. , 2000, Journal of vestibular research : equilibrium & orientation.

[46]  Sarah H. Creem-Regehr,et al.  Absolute Distance Perception to Locations off the Ground Plane , 2007, Perception.

[47]  Johannes Burge,et al.  The vertical horopter is not adaptable, but it may be adaptive. , 2011, Journal of vision.

[48]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[49]  Brian E. Granger,et al.  IPython: A System for Interactive Scientific Computing , 2007, Computing in Science & Engineering.

[50]  O. Sabouraud.  [Space perception]. , 1978, Revue d'oto-neuro-ophtalmologie.