Home Photo Retrieval: Time Matters

Temporal information has been regarded as a key vehicle for sorting and grouping home photos into albums associated with events. While time-based browsing might be adequate for relatively small photo collection, query and retrieval would be very useful to find relevant photos of an event in large collection. In this paper, we propose the use of temporal events for organizing and representing home photos using structured document formalism and hence a new way to retrieve photos of an event using both image content and temporal context. We describe a hierarchical model of temporal events and the algorithm to construct it from a collection of home photos. In particular, we compute metadata of a node from the metadata of its children recursively to facilitate content-based and context-based matching between a query and an event. With semantic content representation extracted using Visual Keywords and Extended Conceptual Graphs, we demonstrate the effectiveness of photo retrieval on 2400 time-stamped heterogeneous home photos with very promising results.

[1]  Yves Chiaramella,et al.  A Model for Multimedia Information Retrieval , 1996 .

[2]  Mingjing Li,et al.  MyPhotos: a system for home photo management and processing , 2002, MULTIMEDIA '02.

[3]  Stephen E. Robertson,et al.  Effective site finding using link anchor information , 2001, SIGIR '01.

[4]  B. Kvasov.  Cubic Spline Interpolation , 2000 .

[5]  Kerry Rodden.  How Do People Organise Their Photographs? , 1999, BCS-IRSG Annual Colloquium on IR Research.

[6]  John R. Smith,et al.  Searching for Images and Videos on the World-Wide Web , 1999 .

[7]  Mounia Lalmas,et al.  Dempster-Shafer's theory of evidence applied to structured documents: modelling uncertainty , 1997, SIGIR '97.

[8]  Ross Wilkinson,et al.  Effective retrieval of structured documents , 1994, SIGIR '94.

[9]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[10]  Kerry Rodden,et al.  How do people manage their digital photographs? , 2003, CHI '03.

[11]  J. C. Platt.  AutoAlbum: clustering digital photographs using probabilistic model merging , 2000, 2000 Proceedings Workshop on Content-based Access of Image and Video Libraries.

[12]  Mary Czerwinski,et al.  PhotoTOC: automatic clustering for browsing personal photographs , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[13]  Ingemar J. Cox,et al.  The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments , 2000, IEEE Trans. Image Process..

[14]  Joo-Hwee Lim,et al.  Symbolic photograph content-based retrieval , 2002, CIKM '02.

[15]  John F. Sowa,et al.  Conceptual Structures: Information Processing in Mind and Machine , 1983 .

[16]  Andreas Paepcke,et al.  Time as essence for photo browsing through personal digital libraries , 2002, JCDL '02.

[17]  Sung-Hyon Myaeng,et al.  A flexible model for retrieval of SGML documents , 1998, SIGIR '98.

[18]  Massimo Marchiori,et al.  The Quest for Correct Information on the Web: Hyper Search Engines , 1997, Comput. Networks.

[19]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .