Perceptual Scale Space and its Applications

In this paper, we study a perceptual scale space by constructing a so-called sketch pyramid which augments the Gaussian and Laplacian pyramid representations in traditional image scale space theory. Each level of this sketch pyramid is a generic attributed graph - called the primal sketch which is inferred from the corresponding image at the same level of the Gaussian pyramid. When images are viewed at increasing resolutions, more details are revealed. This corresponds to perceptual transitions which are represented by topological changes in the sketch graph in terms of a graph grammar. We compute the sketch or perceptual pyramid by Bayesian inference upwards-downwards the pyramid using Markov chain Monte Carlo reversible jumps. We show two example applications of this perceptual scale space: (1) motion tracking of objects over scales, and (2) adaptive image displays which can efficiently show a large high resolution image in a small screen (of a PDA for example) through a selective tour of its image pyramid. Other potential applications include super resolution and multiresolution object recognition.

[1]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[2]  Song-Chun Zhu,et al.  Primal sketch: Integrating structure and texture , 2007, Comput. Vis. Image Underst..

[3]  Modeling complex motion by tracking and editing hidden Markov graphs , 2004, CVPR 2004.

[4]  Tony Lindeberg,et al.  Detecting salient blob-like image structures and their scales with a scale-space primal sketch: A method for focus-of-attention , 1993, International Journal of Computer Vision.

[5]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[6]  Hong Chen,et al.  A high resolution grammatical model for face representation and sketching , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[7]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[8]  Tony Lindeberg,et al.  Feature Detection with Automatic Scale Selection , 1998, International Journal of Computer Vision.

[9]  John M. Gauch,et al.  Multiresolution Analysis of Ridges and Valleys in Grey-Scale Images , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Andrew P. Witkin,et al.  Scale-space filtering: A new approach to multi-scale description , 1984, ICASSP.

[11]  Ying Wu,et al.  Collaborative tracking of multiple targets , 2004, CVPR 2004.

[12]  Song-Chun Zhu,et al.  From Information Scaling of Natural Images to Regimes of Statistical Models , 2007 .

[13]  King-Sun Fu,et al.  Attributed Grammar-A Tool for Combining Syntactic and Statistical Approaches to Pattern Recognition , 1980, IEEE Transactions on Systems, Man, and Cybernetics.

[14]  Andrew P. Witkin,et al.  Scale-Space Filtering , 1983, IJCAI.

[15]  Peter Johansen,et al.  Gaussian Scale-Space Theory , 1997, Computational Imaging and Vision.

[16]  D. Ruderman.  The statistics of natural images , 1994 .

[17]  N. Ahuja,et al.  A transform for detection of multiscale image structure , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Jan-Olof Eklundh,et al.  Scale-space primal sketch: construction and experiments , 1992, Image Vis. Comput..

[19]  Ramakrishnan Srikant,et al.  Fast algorithms for mining association rules , 1998, VLDB 1998.

[20]  Song-Chun Zhu,et al.  Modeling Visual Patterns by Integrating Descriptive and Generative Methods , 2004, International Journal of Computer Vision.

[21]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[22]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[24]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[25]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[26]  B. Julesz.  Textons, the elements of texture perception, and their interactions , 1981, Nature.

[27]  D. Mumford,et al.  Stochastic models for generic images , 2001 .

[28]  Song-Chun Zhu,et al.  Embedding Gestalt Laws in Markov Random Fields , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[30]  Ali Shokoufandeh,et al.  On the Representation and Matching of Qualitative Shape at Multiple Scales , 2002, ECCV.

[31]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[32]  Alan L. Yuille,et al.  FORMS: A flexible object recognition and modelling system , 1996, International Journal of Computer Vision.

[33]  Eero P. Simoncelli,et al.  On Advances in Statistical Modeling of Natural Images , 2004, Journal of Mathematical Imaging and Vision.

[34]  J. Koenderink.  The structure of images , 2004, Biological Cybernetics.

[35]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[36]  Philip N. Klein,et al.  Shape matching using edit-distance: an implementation , 2001, SODA '01.

[37]  Tony Lindeberg.  Edge Detection and Ridge Detection with Automatic Scale Selection , 2004, International Journal of Computer Vision.

[38]  Michael Brady,et al.  Saliency, Scale and Image Description , 2001, International Journal of Computer Vision.

[39]  Mads Nielsen,et al.  Multiscale Gradient Magnitude Watershed Segmentation , 1997, ICIAP.

[40]  Xing Xie,et al.  Browsing large pictures under limited display sizes , 2006, IEEE Transactions on Multimedia.

[41]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[42]  Song-Chun Zhu,et al.  Minimax Entropy Principle and Its Application to Texture Modeling , 1997, Neural Computation.

[43]  Stephen M. Pizer,et al.  A Multiresolution Hierarchical Approach to Image Segmentation Based on Intensity Extrema , 1990, IEEE Trans. Pattern Anal. Mach. Intell..