Perceptual Segmentation: Combining Image Segmentation With Object Tagging

Human observers understand the content of an image intuitively. Based upon image content, they perform many image-related tasks, such as creating slide shows and photo albums, and organizing their image archives. For example, to select photos for an album, people assess image quality based upon the main objects in the image. They modify colors in an image based upon the color of important objects, such as sky, grass or skin. Serious photographers might modify each object separately. Photo applications, in contrast, use low-level descriptors to guide similar tasks. Typical descriptors, such as color histograms, noise level, JPEG artifacts and overall sharpness, can guide an imaging application and safeguard against blunders. However, there is a gap between the outcome of such operations and the same task performed by a person. We believe that the gap can be bridged by automatically understanding the content of the image. This paper presents algorithms for automatic tagging of perceptual objects in images, including sky, skin, and foliage, which constitutes an important step toward this goal.

[1]  Jiebo Luo,et al.  Improved blue sky detection using polynomial model fit , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[2]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[3]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Chun Chen,et al.  A novel Bayesian framework for indoor-outdoor image classification , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[5]  SchmidCordelia,et al.  A Performance Evaluation of Local Descriptors , 2005 .

[6]  Andrea Vedaldi,et al.  Objects in Context , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[7]  Eli Saber,et al.  Unsupervised color image segmentation using a dynamic color gradient thresholding algorithm , 2008, Electronic Imaging.

[8]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[9]  Michel Vidal-Naquet,et al.  Visual features of intermediate complexity and their use in classification , 2002, Nature Neuroscience.

[10]  S. Running,et al.  Remote Sensing of Coniferous Forest Leaf Area , 1986 .

[11]  Eli Saber,et al.  Unsupervised image segmentation by automatic gradient thresholding for dynamic region growth in the CIE L*a*b* color space , 2009, Electronic Imaging.

[12]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[13]  Somchai Jitapunkul,et al.  Face segmentation based on Hue-Cr components and morphological technique , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[14]  Carl Staelin,et al.  Automatic Photo Enhancement Server (HIPIE 2) , 2009 .

[15]  Tetsuo Asano,et al.  Polynomial-time solutions to image segmentation , 1996, SODA '96.

[16]  Thomas Serre,et al.  Categorization by Learning and Combining Object Parts , 2001, NIPS.

[17]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[18]  Antonio Criminisi,et al.  TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context , 2007, International Journal of Computer Vision.

[19]  A. Murat Tekalp,et al.  Automatic Image Annotation Using Adaptive Color Classification , 1996, CVGIP Graph. Model. Image Process..

[20]  Anil K. Jain,et al.  Detecting sky and vegetation in outdoor images , 1999, Electronic Imaging.

[21]  Antonio Criminisi,et al.  TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation , 2006, ECCV.

[22]  Ioannis Pitas,et al.  Face localization and facial feature extraction based on shape and color information , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[23]  Eli Shechtman,et al.  Matching Local Self-Similarities across Images and Videos , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Martin Szummer,et al.  Indoor-outdoor image classification , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[25]  Mark Q. Shaw,et al.  Automatic Image Segmentation by Dynamic Region Growth and Multiresolution Merging , 2009, IEEE Transactions on Image Processing.

[26]  Shigeru Akamatsu,et al.  Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[27]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[28]  Jiebo Luo,et al.  A physical model-based approach to detecting sky in photographic images , 2002, IEEE Trans. Image Process..

[29]  Christos Grecos,et al.  A fast skin region detector for colour images , 2005 .

[30]  J JonesMichael,et al.  Statistical color models with application to skin detection , 2002 .

[31]  Eli Saber,et al.  Automatic color image segmentation by dynamic region growth and multimodal merging of color and texture information , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[32]  Anil K. Jain,et al.  On image classification: city images vs. landscapes , 1998, Pattern Recognit..