Video clustering based on the collaboration of multimedia clusterers

This paper addresses the task of video clustering, representing the videos as complex multimedia objects, which are an aggregation of heterogeneous data (text, images and audio) as a single unit. In support to the development of this task, this paper proposes a new method based on the collaboration of clusterers evaluating the multimedia resources present in videos. The new method, called Multimedia Collaborative Multi-clustering, is evaluated using the video set of MediaEval 2011, implementing a text clusterer with the speech transcription provided by the video set, an image clusterer with keyframes also provided by the video set, and an audio clusterer with the acoustic signal of the videos. A comparison against several solutions found in the literature demonstrates the feasibility of the proposed method, which creates clustering structures closer to the actual classification of the videos than the clusters produced by other solutions.

[1]  Germain Forestier,et al.  Towards conflict resolution in collaborative clustering , 2010, 2010 5th IEEE International Conference Intelligent Systems.

[2]  Pierre Gançarski,et al.  Collaborative multi-step mono-level multi-strategy classification , 2007, Multimedia Tools and Applications.

[3]  Mohammad Soleymani,et al.  Automatic tagging and geotagging in video collections and communities , 2011, ICMR.

[4]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[5]  Anil K. Jain,et al.  Text information extraction in images and video: a survey , 2004, Pattern Recognit..

[6]  Leo Galambos.  Semi-automatic stemmer evaluation , 2004, Intelligent Information Systems.

[7]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[8]  Yi Yang,et al.  Harmonizing Hierarchical Manifolds for Multimedia Document Semantics Understanding and Cross-Media Retrieval , 2008, IEEE Transactions on Multimedia.

[9]  Zhu Liu,et al.  Multimedia content analysis-using both audio and visual clues , 2000, IEEE Signal Process. Mag..

[10]  Ada Wai-Chee Fu,et al.  Incremental Document Clustering for Web Page Classification , 2002 .

[11]  L. Hubert,et al.  Comparing partitions , 1985 .

[12]  Qi Tian,et al.  News video search with fuzzy event clustering using high-level features , 2006, MM '06.

[13]  Ana L. N. Fred,et al.  Analysis of consensus partition in cluster ensemble , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[14]  Ara V. Nefian,et al.  Web Image Clustering , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[15]  Joachim M. Buhmann,et al.  Bagging for Path-Based Clustering , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Carla E. Brodley,et al.  Feature Selection for Unsupervised Learning , 2004, J. Mach. Learn. Res..

[17]  James Bailey,et al.  COALA: A Novel Approach for the Extraction of an Alternate Clustering of High Quality and High Dissimilarity , 2006, Sixth International Conference on Data Mining (ICDM'06).

[18]  Oscar Ortega,et al.  Clustering Complex Multimedia Objects using an Ensemble Approach , 2012, ICPRAM.

[19]  Gabriela Csurka,et al.  Introduction to the special issue on “metadata mining for image understanding” , 2008, Multimedia Tools and Applications.

[20]  Aimin Yang,et al.  A KFCM-Based Fuzzy Classifier , 2007, Fourth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD 2007).

[21]  Honggang Zhang,et al.  Web Multimedia Object Clustering via Information Fusion , 2011, 2011 International Conference on Document Analysis and Recognition.

[22]  Diane J. Cook,et al.  Using Closed Captions and Visual Features to Classify Movies by Genre , 2006 .

[23]  Anil K. Jain,et al.  Multiobjective data clustering , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[24]  Rich Caruana,et al.  Meta Clustering , 2006, Sixth International Conference on Data Mining (ICDM'06).

[25]  Thomas Sikora,et al.  Feature-based video key frame extraction for low quality video sequences , 2009, 2009 10th Workshop on Image Analysis for Multimedia Interactive Services.

[26]  Gunter Saake,et al.  A schema matching-based approach to XML schema clustering , 2008, iiWAS.

[27]  Abdolreza Mirzaei,et al.  A Novel Hierarchical-Clustering-Combination Scheme Based on Fuzzy-Similarity Relations , 2010, IEEE Transactions on Fuzzy Systems.

[28]  Martha Larson,et al.  Overview of MediaEval 2011 Rich Speech Retrieval Task and Genre Tagging Task , 2011, MediaEval.

[29]  Yi Yang,et al.  Mining Semantic Correlation of Heterogeneous Multimedia Data for Cross-Media Retrieval , 2008, IEEE Transactions on Multimedia.

[30]  Germain Forestier,et al.  Mining Multiple Satellite Sensor Data Using Collaborative Clustering , 2009, 2009 IEEE International Conference on Data Mining Workshops.

[31]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[32]  Ian Davidson,et al.  Finding Alternative Clusterings Using Constraints , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[33]  Rama Chellappa,et al.  Pattern Recognition in Video , 2005, PReMI.

[34]  Mathias Lux,et al.  Lire: lucene image retrieval: an extensible java CBIR library , 2008, ACM Multimedia.

[35]  Shih-Fu Chang,et al.  Clustering methods for video browsing and annotation , 1996, Electronic Imaging.

[36]  Gerardo Beni,et al.  A Validity Measure for Fuzzy Clustering , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Sudeep Sarkar,et al.  Audio Segmentation and Speaker Localization in Meeting Videos , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[38]  Ana L. N. Fred,et al.  Combining multiple clusterings using evidence accumulation , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Germain Forestier,et al.  Multisource Images Analysis Using Collaborative Clustering , 2008, EURASIP J. Adv. Signal Process..

[40]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[41]  Anil K. Jain,et al.  Clustering ensembles: models of consensus and weak partitions , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Jane Hunter,et al.  Implementing Preservation Strategies for Complex Multimedia Objects , 2003, ECDL.

[43]  H. Charles Romesburg,et al.  Cluster analysis for researchers , 1984 .

[44]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[45]  Mohamed S. Kamel,et al.  Cumulative Voting Consensus Method for Partitions with Variable Number of Clusters , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Joan Claudi Socoró,et al.  Positional and confidence voting-based consensus functions for fuzzy cluster ensembles , 2012, Fuzzy Sets Syst..

[47]  Ichiro Fujinaga,et al.  jAudio: An Feature Extraction Library , 2005, ISMIR.

[48]  Jiaheng Lu,et al.  Clustering Web video search results based on integration of multiple features , 2010, World Wide Web.

[49]  Francisco de A. T. de Carvalho,et al.  Fuzzy c-means clustering methods for symbolic interval data , 2007, Pattern Recognit. Lett..

[50]  Christoph F. Eick,et al.  An architecture and algorithms for multi-run clustering , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[51]  Sandrine Dudoit,et al.  Bagging to Improve the Accuracy of A Clustering Procedure , 2003, Bioinform..