Feature synthesized EM algorithm for image retrieval

As a commonly used unsupervised learning algorithm in Content-Based Image Retrieval (CBIR), Expectation-Maximization (EM) algorithm has several limitations, including the curse of dimensionality and the convergence at a local maximum. In this article, we propose a novel learning approach, namely Coevolutionary Feature Synthesized Expectation-Maximization (CFS-EM), to address the above problems. The CFS-EM is a hybrid of coevolutionary genetic programming (CGP) and EM algorithm applied on partially labeled data. CFS-EM is especially suitable for image retrieval because the images can be searched in the synthesized low-dimensional feature space, while a kernel-based method has to make classification computation in the original high-dimensional space. Experiments on real image databases show that CFS-EM outperforms Radial Basis Function Support Vector Machine (RBF-SVM), CGP, Discriminant-EM (D-EM) and Transductive-SVM (TSVM) in the sense of classification performance and it is computationally more efficient than RBF-SVM in the query phase.

[1]  Thomas S. Huang,et al.  Water-filling: a novel way for image structural feature extraction , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[2]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[3]  Virginia R. de Sa,et al.  Learning Classification with Unlabeled Data , 1993, NIPS.

[4]  Gunnar Rätsch,et al.  An Introduction to Boosting and Leveraging , 2002, Machine Learning Summer School.

[5]  P. Deb Finite Mixture Models , 2008 .

[6]  B. S. Manjunath,et al.  Dimensionality reduction for image retrieval , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[7]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[8]  Bir Bhanu,et al.  Evolutionary feature synthesis for object recognition , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[10]  Sebastian Thrun,et al.  Text Classification from Labeled and Unlabeled Documents using EM , 2000, Machine Learning.

[11]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[12]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Feiping Nie,et al.  Texture Image Segmentation: An Interactive Framework Based on Adaptive Features and Transductive Learning , 2006, ACCV.

[14]  John R. Koza,et al.  Genetic programming 2 - automatic discovery of reusable programs , 1994, Complex Adaptive Systems.

[15]  Bir Bhanu,et al.  Genetic algorithm based feature selection for target detection in SAR images , 2003, Image Vis. Comput..

[16]  Newton Lee,et al.  ACM Transactions on Multimedia Computing, Communications and Applications (ACM TOMCCAP) , 2007, CIE.

[17]  Bir Bhanu,et al.  Improving retrieval performance by long-term relevance information , 2002, Object recognition supported by user interaction for service robots.

[18]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Yuxiao Hu,et al.  Learning a locality preserving subspace for visual recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[20]  David Bargeron,et al.  Boosting-based transductive learning for text detection , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[21]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[22]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[23]  Lei Zhu,et al.  Supporting multi-example image queries in image databases , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[24]  Bir Bhanu,et al.  Probabilistic Feature Relevance Learning for Content-Based Image Retrieval , 1999, Comput. Vis. Image Underst..

[25]  Stan Z. Li,et al.  Extraction of feature subspaces for content-based retrieval using relevance feedback , 2001, MULTIMEDIA '01.

[26]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[27]  T. Kanade,et al.  Genetic Learning For Adaptive Image Segmentation , 1994 .

[28]  Qi Tian,et al.  Learning image manifolds by semantic subspace projection , 2006, MM '06.

[29]  Una-May O'Reilly,et al.  Genetic Programming II: Automatic Discovery of Reusable Programs. , 1994, Artificial Life.

[30]  Bir Bhanu,et al.  Integrating relevance feedback techniques for image retrieval using reinforcement learning , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[32]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[33]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Bir Bhanu,et al.  Evolutionary Feature Synthesis for Image Databases , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[35]  Qi Tian,et al.  Discriminant-EM algorithm with application to image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[36]  Thomas S. Huang,et al.  3D MARS: immersive virtual reality for content-based image retrieval , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[37]  Thorsten Joachims,et al.  Transductive Learning via Spectral Graph Partitioning , 2003, ICML.

[38]  Wei-Ying Ma,et al.  Learning an image manifold for retrieval , 2004, MULTIMEDIA '04.

[39]  B. Bhanu,et al.  Coevolutionary feature synthesized EM algorithm for image retrieval , 2005, MULTIMEDIA '05.

[40]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Bir Bhanu,et al.  A new semi-supervised EM algorithm for image retrieval , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[42]  Rudolf Hanka,et al.  Histological image retrieval based on semantic content analysis , 2003, IEEE Transactions on Information Technology in Biomedicine.

[43]  Yuchun Fang,et al.  Experiments in Mental Face Retrieval , 2005, AVBPA.

[44]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Juyang Weng,et al.  Hierarchical Discriminant Analysis for Image Retrieval , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Chen Yi A Progressive Transductive Inference Algorithm Based on Support Vector Machine , 2003 .

[47]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[48]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[49]  Ying Wu,et al.  Color tracking by transductive learning , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[50]  Bir Bhanu,et al.  Active concept learning in image databases , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[51]  Alexander Gammerman,et al.  Transduction with Confidence and Credibility , 1999, IJCAI.