Hierarchical photo stream segmentation using context

Photo stream segmentation is to segment photo streams into groups, each of which corresponds to an event. Photo stream segmentation can be done with or without prior knowledge of event structure. In this paper, we study the problem by assuming that there is no a priori event model available. Although both context and content information are important for photo stream segmentation, we focus on investigating the usage of context information in this work. We consider different information components of context such as time, location, and optical setting for inexpensive segmentation of photo streams from common users of modern digital camera. As events are hierarchical, we propose to segment photo stream using hierarchical mixture model. We compare the generated hierarchy with that created by users to see how well results can be obtained without knowing the prior event model. We experimented with about 3000 photos from amateur photographers to study the efficacy of the approach for these context information components.

[1]  Gordon Bell,et al.  MyLifeBits: fulfilling the Memex vision , 2002, MULTIMEDIA '02.

[2]  Marc Gelgon,et al.  Incremental statistical geo-temporal structuring of a personal camera phone image collection , 2004, ICPR 2004.

[3]  Dan Schonfeld,et al.  Statistical sequential analysis for real-time video scene change detection on compressed multimedia bitstream , 2003, IEEE Trans. Multim..

[4]  Ramesh Jain,et al.  Event Discovery in Multimedia Reconnaissance Data Using Spatio-Temporal Clustering , 2006 .

[5]  Ramesh Jain,et al.  Segmenting Photo Streams in Events Based on Optical Metadata , 2007 .

[6]  Andreas Paepcke,et al.  Time as essence for photo browsing through personal digital libraries , 2002, JCDL '02.

[7]  Andreas E. Savakis,et al.  Automatic image event segmentation and quality screening for albuming applications , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[8]  Andreas Girgensohn,et al.  Temporal event clustering for digital photo collections , 2003, ACM Multimedia.

[9]  Kiyoharu Aizawa,et al.  Context-based video retrieval system for the life-log applications , 2003, MIR '03.

[10]  Alan F. Smeaton,et al.  Multimodal Segmentation of Lifelog Data , 2007, RIAO.

[11]  Ramin Zabih,et al.  A feature-based algorithm for detecting and classifying production effects , 1999, Multimedia Systems.

[12]  Marc Gelgon,et al.  Building and tracking hierarchical geographical & temporal partitions for image collection management on mobile devices , 2005, MULTIMEDIA '05.

[13]  Wei-Hao Lin,et al.  Structuring continuous video recordings of everyday life using time-constrained clustering , 2006, Electronic Imaging.

[14]  Boon-Lock Yeo,et al.  Segmentation of Video by Clustering and Graph Analysis , 1998, Comput. Vis. Image Underst..

[15]  Mary Czerwinski,et al.  PhotoTOC: automatic clustering for browsing personal photographs , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[16]  John R. Kender,et al.  Video scene segmentation via continuous video coherence , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[17]  Ramesh C. Jain,et al.  Digital video segmentation , 1994, MULTIMEDIA '94.

[18]  Marcel Worring,et al.  Systematic evaluation of logical story unit segmentation , 2002, IEEE Trans. Multim..

[19]  Boon-Lock Yeo,et al.  Time-constrained clustering for segmentation of video into story units , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[20]  John S. Boreczky,et al.  Comparison of video shot boundary detection techniques , 1996, Electronic Imaging.

[21]  Keansub Lee,et al.  Minimal-impact audio-based personal archives , 2004, CARPE'04.

[22]  J. C. Platt.  AutoAlbum: clustering digital photographs using probabilistic model merging , 2000, 2000 Proceedings Workshop on Content-based Access of Image and Video Libraries.

[23]  H. Garcia-Molina,et al.  Automatic organization for digital photographs with geographic coordinates , 2004, Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries, 2004..

[24]  Mubarak Shah,et al.  Video scene segmentation using Markov chain Monte Carlo , 2006, IEEE Transactions on Multimedia.

[25]  Svetha Venkatesh,et al.  Extraction of social context and application to personal multimedia exploration , 2006, MM '06.