ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , VOL . ? ? , NO . ? ? , ? ? ? 201 ? 1 eXclusive Component Analysis : Theory and Applications

an enhancement of the Independent Component Analysis (ICA), named eXclusive Component Analysis (XCA) and its applications on image statistics and machine learning is presented. XCA is especially effective when comparing the characteristics of several datasets. This effectiveness arise from XCA’s ability to find exclusive features of each dataset as well as features common to all datasets. This paper also present a new set of classification methods, XCABoost, that are developed to utilize the exclusive components of datasets elucidated by XCA. XCABoost methods are formulated as an ensemble of linear and non-linear classifiers and as a classifier for Multiple Instance Learning. The XCABoost methods is benchmarked using well known datasets and compared to 27 other classifiers with good performance.

[1]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[2]  Erkki Oja,et al.  A fast algorithm for estimating overcomplete ICA bases for image windows , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[3]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[4]  Aapo Hyvärinen,et al.  Topographic ICA as a model of V1 receptive fields , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[5]  J. Nadal.  Non linear neurons in the low noise limit : a factorial code maximizes information transferJean , 1994 .

[6]  Thomas G. Dietterich,et al.  Solving the Multiple Instance Problem with Axis-Parallel Rectangles , 1997, Artif. Intell..

[7]  Thomas Gärtner,et al.  Multi-Instance Kernels , 2002, ICML.

[8]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[9]  Liqing Zhang,et al.  Overcomplete topographic independent component analysis , 2008, Neurocomputing.

[10]  Jun Zhou,et al.  MILIS: Multiple Instance Learning with Instance Selection , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Aapo Hyvärinen,et al.  Estimating Overcomplete Independent Component Bases for Image Windows , 2002, Journal of Mathematical Imaging and Vision.

[12]  Daniel D. Lee,et al.  Multiplicative Updates for Nonnegative Quadratic Programming in Support Vector Machines , 2002, NIPS.

[13]  Boris Babenko.  Multiple Instance Learning: Algorithms and Applications , 2008 .

[14]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[15]  Aapo Hyvärinen,et al.  Topographic ICA as a Model of Natural Image Statistics , 2000, Biologically Motivated Computer Vision.

[16]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[17]  Wu-Jun Li,et al.  MILD: Multiple-Instance Learning via Disambiguation , 2010, IEEE Transactions on Knowledge and Data Engineering.

[18]  Paul A. Viola,et al.  Multiple Instance Boosting for Object Detection , 2005, NIPS.

[19]  Thomas Hofmann,et al.  Support Vector Machines for Multiple-Instance Learning , 2002, NIPS.

[20]  Yan Nei Law,et al.  Automated Protein Distribution Detection in High-Throughput Image-Based siRNA Library Screens , 2009, J. Signal Process. Syst..

[21]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[22]  Jun Wang,et al.  Solving the Multiple-Instance Problem: A Lazy Learning Approach , 2000, ICML.

[23]  A. Meyer-Base,et al.  Topographic independent component analysis for fMRI signal detection , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[24]  Zhi-Hua Zhou,et al.  Multi-instance learning by treating instances as non-I.I.D. samples , 2008, ICML '09.

[25]  Aapo Hyvärinen,et al.  A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images , 2001, Vision Research.

[26]  Dinh Tuan Pham,et al.  Separation of a mixture of independent sources through a maximum likelihood approach , 1992 .

[27]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[28]  Qi Zhang,et al.  EM-DD: An Improved Multiple-Instance Learning Technique , 2001, NIPS.

[29]  P. Comon.  Independent Component Analysis , 1992 .

[30]  Peter V. Gehler,et al.  Deterministic Annealing for Multiple-Instance Learning , 2007, AISTATS.

[31]  Yixin Chen,et al.  MILES: Multiple-Instance Learning via Embedded Instance Selection , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Aapo Hyvärinen,et al.  Natural Image Statistics - A Probabilistic Approach to Early Computational Vision , 2009, Computational Imaging and Vision.

[33]  Ivor W. Tsang,et al.  A Convex Method for Locating Regions of Interest with Multi-instance Learning , 2009, ECML/PKDD.

[34]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[35]  James T. Kwok,et al.  Online multiple instance learning with no regret , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  Aykut Erdem,et al.  Multiple-Instance Learning with Instance Selection via Dominant Sets , 2011, SIMBAD.

[37]  I. Miller.  Probability, Random Variables, and Stochastic Processes , 1966 .

[38]  Lukasz A. Kurgan,et al.  mi-DS: Multiple-Instance Learning Algorithm , 2013, IEEE Transactions on Cybernetics.

[39]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[40]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[41]  Yixin Chen,et al.  Image Categorization by Learning and Reasoning with Regions , 2004, J. Mach. Learn. Res..