Psychophysical study of image orientation perception

This paper presents a psychophysical study on the perception of image orientation. Some natural images are extremely difficult even for humans to orient correctly or may not even have a "correct" orientation; the study provides an upper bound for the performance of an automatic system. Discrepant detection rates based on only low-level cues have been reported, ranging from exceptionally high in earlier work to more reasonable in recent work. This study allows us to put the reported results in the correct perspective. In addition, the use of a large, carefully chosen image set that spans the "photo space" (in terms of occasions and subject matter) and extensive interaction with the human observers should reveal cues used by humans at various image resolutions. These can be used to design a robust automatic algorithm for orientation detection. A collection of 1000 images (mix of professional photos and consumer snapshots) is used in this study. Each image is examined by at least five observers and shown at varying resolutions. Object recognition is expected to be more difficult (impossible for some images) at the lowest resolution and easier as the resolution increases. At each resolution, observers are asked to indicate the image orientation, the level of confidence, and the cues they used to make the decision. This study suggests that for typical images, the upper bound on accuracy is close to 98% when using all available semantic cues from high-resolution images and 84% if only low-level vision features and coarse semantics from thumbnails are used. The study also shows that sky and people are the most useful and reliable among a number of important semantic cues.

[1]  L. Braine,et al.  On how adults identify the orientation of a shape , 1981, Perception & psychophysics.

[2]  R. Maki.  Naming and locating the tops of rotated pictures. , 1986, Canadian journal of psychology.

[3]  R. Hess,et al.  The interaction of first- and second-order cues to orientation , 1999, Vision Research.

[4]  R. Shapley,et al.  Contextual influences on orientation discrimination: binding local and global cues , 2001, Vision Research.

[5]  Anil K. Jain,et al.  Automatic image orientation detection , 2002, IEEE Trans. Image Process..

[6]  Elinor McKone,et al.  Orientation invariance in naming rotated objects: Individual differences and repetition priming , 1999, Perception & psychophysics.

[7]  Takeo Kanade,et al.  A statistical method for 3D object detection applied to faces and cars , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[8]  A. Murat Tekalp,et al.  Automatic Image Annotation Using Adaptive Color Classification , 1996, CVGIP Graph. Model. Image Process..

[9]  M. Corballis,et al.  Decisions about identity and orientation of rotated letters and digits , 1978, Memory & cognition.

[10]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Neill W. Campbell,et al.  Automatic Segmentation and Classification of Outdoor Images Using Neural Networks , 1997, Int. J. Neural Syst..

[12]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[13]  Adam Reeves,et al.  The use of word-picture verification to study entry-level object recognition: Further support for view-invariant mechanisms , 2002, Memory & cognition.

[14]  Michael J. Tarr.  Is human object recognition better described by geon structural description or by multiple views , 1995 .

[15]  P. Jolicoeur.  The time to name disoriented natural objects , 1985, Memory & cognition.

[16]  Pierre Jolicoeur,et al.  Recognition thresholds for plane-rotated pictures of familiar objects. , 2003, Acta psychologica.

[17]  Anil K. Jain,et al.  On image classification: city images vs. landscapes , 1998, Pattern Recognit..

[18]  Adam Reeves,et al.  Rotating objects to determine orientation, not identity: Evidence from a backward-masking/dual-task procedure , 2000, Perception & psychophysics.

[19]  Stefano A. De Caro.  On the perception of objects and their orientations. , 1998 .

[20]  Jiebo Luo,et al.  A physical model-based approach to detecting sky in photographic images , 2002, IEEE Trans. Image Process..

[21]  Jake K. Aggarwal,et al.  Combining structure, color and texture for image retrieval: A performance evaluation , 2002, Object recognition supported by user interaction for service robots.

[22]  Martin Szummer,et al.  Indoor-outdoor image classification , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[23]  Manila Vannucci,et al.  Category Effects on the Processing of Plane-Rotated Objects , 2000, Perception.

[24]  Jake K. Aggarwal,et al.  Retrieval by classification of images containing large manmade objects using perceptual grouping , 2002, Pattern Recognit..

[25]  Takeo Kanade,et al.  A statistical approach to 3d object detection applied to faces and cars , 2000 .

[26]  Heinrich H Bülthoff,et al.  Image-based object recognition in man, monkey and machine , 1998, Cognition.

[27]  Takeo Kanade,et al.  Probabilistic modeling of local appearance and spatial relationships for object recognition , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[28]  Yongmei Wang,et al.  Content-based image orientation detection with support vector machines , 2001, Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL 2001).

[29]  Toby J. Lloyd-Jones,et al.  Effects of plane rotation, task, and complexity on recognition of familiar and chimeric objects , 2002, Memory & cognition.

[30]  Milind R. Naphade,et al.  A probabilistic framework for semantic indexing and retrieval in video , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[31]  Pierre Jolicoeur,et al.  The influence of perceived rotary motion on the recognition of rotated objects , 1998 .

[32]  G K Humphrey,et al.  Surface Cues Reduce the Latency to Name Rotated Images of Objects , 2001, Perception.

[33]  Jiebo Luo,et al.  Indoor vs outdoor classification of consumer photographs using low-level and semantic features , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[34]  P. Mcmullen,et al.  Effects of orientation on the identification of rotated objects depend on the level of identity. , 1998, Journal of experimental psychology. Human perception and performance.

[35]  P Jolicoeur,et al.  Orientation congruency effects in visual search. , 1992, Canadian journal of psychology.