Photobook: Content-based manipulation of image databases

We describe the Photobook system, which is a set of interactive tools for browsing and searching images and image sequences. These query tools differ from those used in standard image databases in that they make direct use of the image content rather than relying on text annotations. Direct search on image content is made possible by use of semantics-preserving image compression, which reduces images to a small set of perceptually-significant coefficients. We discuss three types of Photobook descriptions in detail: one that allows search based on appearance, one that uses 2-D shape, and a third that allows search based on textural properties. These image content descriptions can be combined with each other and with text-based descriptions to provide a sophisticated browsing and search capability. In this paper we demonstrate Photobook on databases containing images of people, video keyframes, hand tools, fish, texture swatches, and 3-D medical data.

[1]  H. Helson,et al.  Prediction theory and Fourier Series in several variables , 1958 .

[2]  H. Helson,et al.  Prediction theory and fourier series in several variables. II , 1961 .

[3]  Phil Brodatz,et al.  Textures: A Photographic Album for Artists and Designers , 1966 .

[4]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[5]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[6]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[7]  Yehezkel Lamdan,et al.  Geometric Hashing: A General And Efficient Model-based Recognition Scheme , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[8]  William I. Grosky,et al.  Shape matching utilizing indexed hypotheses generation and testing , 1989, IEEE Trans. Robotics Autom..

[9]  Andrew Lippman,et al.  Coding image sequences for interactive retrieval , 1989, CACM.

[10]  Juha Röning,et al.  Algorithms and Architectures for Machine Vision , 1989 .

[11]  Suh-Yin Lee,et al.  2D C-string: A new spatial knowledge representation for image database systems , 1990, Pattern Recognit..

[12]  Satoshi Tanaka,et al.  Retrieval Method For An Image Database Based On Topological Structure , 1990, Optics & Photonics.

[13]  S. Tanaka,et al.  An intelligent user interface to an image database using a figure interpretation method , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[14]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[15]  Alex Pentland,et al.  Closed-Form Solutions for Physically Based Shape Modeling and Recognition , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Raimondo Schettini,et al.  Indexing and Fuzzy Logic-Based Retrieval of Color Images , 1991, Visual Database Systems.

[17]  Costas Xydeas,et al.  Classification of shape for content retrieval of images in a multimedia database , 1991 .

[18]  Patrick Campbell McLean,et al.  Structured video coding , 1991 .

[19]  Michael S. Landy,et al.  Computational models of visual processing , 1991 .

[20]  Alex Pentland,et al.  Closed-form solutions for physically-based shape modeling and recognition , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  A. Pentland,et al.  Robust estimation of a multi-layered motion representation , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[22]  Toshikazu Kato,et al.  A cognitive approach to visual interaction , 1991 .

[23]  H. V. Jagadish,et al.  A retrieval technique for similar shapes , 1991, SIGMOD '91.

[24]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[25]  Zen Chen,et al.  Computer vision for robust 3D aircraft recognition with fast library search , 1991, Pattern Recognit..

[26]  Suh-Yin Lee,et al.  Retrieval of similar pictures on pictorial databases , 1991, Pattern Recognit..

[27]  Anil K. Jain,et al.  Texture classification and segmentation using multiresolution simultaneous autoregressive models , 1992, Pattern Recognit..

[28]  Charles W. Therrien,et al.  Discrete Random Signals and Statistical Signal Processing , 1992 .

[29]  SUH-YIN LEE,et al.  Spatial reasoning and similarity retrieval of images using 2D C-string knowledge representation , 1992, Pattern Recognit..

[30]  Chin-Chen Chang,et al.  Retrieving the Most Similar Symbolic Pictures from Pictorial Databases , 1992, Inf. Process. Manag..

[31]  William I. Grosky,et al.  A pictorial index mechanism for model-based matching , 1992, Data Knowl. Eng..

[32]  Toshikazu Kato,et al.  Query by Visual Example - Content based Image Retrieval , 1992, EDBT.

[33]  Calyampudi Radhakrishna Rao,et al.  Signal Processing and its Applications , 1993, Handbook of Statistics.

[34]  Ramesh C. Jain.  NSF workshop on Visual Information Management Systems , 1993, SGMD.

[35]  Rosalind W. Picard,et al.  Finding similar patterns in large image databases , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[36]  Edward H. Adelson,et al.  Layered representation for motion analysis , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Christos Faloutsos,et al.  QBIC project: querying images by content, using color, texture, and shape , 1993, Electronic Imaging.

[38]  Gerald L. Lohse,et al.  Towards a texture naming system: Identifying relevant dimensions of texture , 1993, Vision Research.

[39]  Alex Pentland,et al.  A modal framework for correspondence and description , 1993, 1993 (4th) International Conference on Computer Vision.

[40]  Alex Pentland,et al.  Face recognition using view-based and modular eigenspaces , 1994, Optics & Photonics.

[41]  Stephen W. Smoliar,et al.  Content based video indexing and retrieval , 1994, IEEE MultiMedia.

[42]  Alex Pentland,et al.  Shape analysis of brain structures using physical and experimental modes , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Rosalind W. Picard,et al.  Finding perceptually dominant orientations in natural textures. , 1994, Spatial vision.

[45]  Fang Liu,et al.  A new Wold ordering for image similarity , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[46]  Alex Pentland,et al.  Modal Matching for Correspondence and Recognition , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Alex Pentland,et al.  Recursive Estimation of Motion, Structure, and Focal Length , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  A. Ravishankar Rao,et al.  Towards a texture naming system: Identifying relevant dimensions of texture , 1993, Vision Research.

[49]  William A. Pearlman,et al.  Texture coding using a Wold decomposition model , 1996, IEEE Trans. Image Process..

[50]  Trevor Darrell,et al.  A novel environment for situated vision and behavior , 1994 .

[51]  Reinhard Wilhelm,et al.  Shape Analysis , 2000, CC.

[52]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[53]  Tom Minka,et al.  Vision texture for annotation , 1995, Multimedia Systems.