Intelligent photo clustering with user interaction and distance metric learning

Photo clustering is an effective way to organize albums and it is useful in many applications, such as photo browsing and tagging. But automatic photo clustering is not an easy task due to the large variation of photo content. In this paper, we propose an interactive photo clustering paradigm that jointly explores human and computer. In this paradigm, the photo clustering task is semi-automatically accomplished: users are allowed to manually adjust clustering results with different operations, such as splitting clusters, merging clusters and moving photos from one cluster to another. Behind users' operations, we have a learning engine that keeps updating the distance measurements between photos in an online way, such that better clustering can be performed based on the distance measure. Experimental results on multiple photo albums demonstrated that our approach is able to improve automatic photo clustering results, and by exploring distance metric learning, our method is much more effective than pure manual adjustments of photo clustering.

[1]  Shiri Gordon,et al.  Unsupervised image-set clustering using an information theoretic framework , 2006, IEEE Transactions on Image Processing.

[2]  Michael I. Jordan,et al.  Learning Spectral Clustering , 2003, NIPS.

[3]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[4]  Meng Wang,et al.  Unified Video Annotation via Multigraph Learning , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Wei-Ying Ma,et al.  Hierarchical clustering of WWW image search results using visual, textual and link information , 2004, MULTIMEDIA '04.

[6]  Tao Mei,et al.  Probabilistic Multimodality Fusion for Event based Home Photo Clustering , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[7]  Yiannis Kompatsiaris,et al.  ClustTour: city exploration by use of hybrid photo clustering , 2010, ACM Multimedia.

[8]  Yi Yang,et al.  Image Clustering Using Local Discriminant Models and Global Integration , 2010, IEEE Transactions on Image Processing.

[9]  Yuandong Tian,et al.  A Face Annotation Framework with Partial Clustering and Interactive Labeling , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  J. C. Platt.  AutoAlbum: clustering digital photographs using probabilistic model merging , 2000, 2000 Proceedings Workshop on Content-based Access of Image and Video Libraries.

[11]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Wei Liu,et al.  Semi-supervised distance metric learning for Collaborative Image Retrieval , 2008, CVPR.

[13]  Ramesh C. Jain,et al.  Personal photo album summarization , 2009, MM '09.

[14]  Dong Liu,et al.  Smart batch tagging of photo albums , 2009, MM '09.

[15]  Tomer Hertz,et al.  Learning a Mahalanobis Metric from Equivalence Constraints , 2005, J. Mach. Learn. Res..

[16]  Lei Zhang,et al.  IGroup: presenting web image search results in semantic clusters , 2007, CHI.

[17]  Mor Naaman,et al.  Generating diverse and representative image search results for landmarks , 2008, WWW.

[18]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Neill W Campbell,et al.  IEEE International Conference on Computer Vision and Pattern Recognition , 2008 .

[20]  Meng Wang,et al.  Active learning in multimedia annotation and retrieval: A survey , 2011, TIST.

[21]  Meng Wang,et al.  Beyond Distance Measurement: Constructing Neighborhood Similarity for Video Annotation , 2009, IEEE Transactions on Multimedia.

[22]  Benjamin B. Bederson,et al.  Semi-Automatic Image Annotation Using Event and Torso Identification , 2004 .

[23]  Ivor W. Tsang,et al.  Dynamic vehicle routing with stochastic requests , 2003, IJCAI 2003.

[24]  Steven M. Seitz,et al.  Scene Summarization for Online Image Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[25]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[26]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[27]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[28]  Andreas Girgensohn,et al.  Temporal event clustering for digital photo collections , 2003, ACM Multimedia.

[29]  Guillaume Pitel,et al.  Image clustering based on a shared nearest neighbors approach for tagged collections , 2008, CIVR '08.

[30]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[31]  Ali Ghodsi,et al.  Distance metric learning vs. Fisher discriminant analysis , 2008, AAAI 2008.

[32]  Chris Ding.  A Tutorial on Spectral Clustering , 2004, ICML 2004.

[33]  Martine D. F. Schlag,et al.  Spectral K-Way Ratio-Cut Partitioning and Clustering , 1993, 30th ACM/IEEE Design Automation Conference.

[34]  Ali Ghodsi,et al.  Distance Metric Learning Versus Fisher Discriminant Analysis , 2008, AAAI.

[35]  Mary Czerwinski,et al.  PhotoTOC: automatic clustering for browsing personal photographs , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[36]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .