On Measure Transformed Canonical Correlation Analysis

In this paper, linear canonical correlation analysis (LCCA) is generalized by applying a structured transform to the joint probability distribution of the considered pair of random vectors, i.e., a transformation of the joint probability measure defined on their joint observation space. This framework, called measure transformed canonical correlation analysis (MTCCA), applies LCCA to the data after transformation of the joint probability measure. We show that judicious choice of the transform leads to a modified canonical correlation analysis, which, in contrast to LCCA, is capable of detecting non-linear relationships between the considered pair of random vectors. Unlike kernel canonical correlation analysis, where the transformation is applied to the random vectors, in MTCCA the transformation is applied to their joint probability distribution. This results in performance advantages and reduced implementation complexity. The proposed approach is illustrated for graphical model selection in simulated data having non-linear dependencies, and for measuring long-term associations between companies traded in the NASDAQ and NYSE stock markets.

[1]  Kellen Petersen August.  Real Analysis , 2009 .

[2]  A. Wald,et al.  On Stochastic Limit and Order Relationships , 1943 .

[3]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[4]  Kazuyuki Aihara,et al.  Detecting generalized synchronization between chaotic signals: a kernel-based approach , 2005 .

[5]  Shotaro Akaho,et al.  A kernel method for canonical correlation analysis , 2006, ArXiv.

[6]  Hassan S. Bakouch,et al.  Elements of Distribution Theory , 2007 .

[7]  Arie Yeredor,et al.  Blind MIMO identification using the second characteristic function , 2004, IEEE Transactions on Signal Processing.

[8]  Michael Elad,et al.  Pixels that sound , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[9]  Anirban DasGupta,et al.  Fundamentals of Probability: A First Course , 2010 .

[10]  Xuelong Li,et al.  A survey of graph edit distance , 2010, Pattern Analysis and Applications.

[11]  Vince D. Calhoun,et al.  Joint Blind Source Separation by Multiset Canonical Correlation Analysis , 2009, IEEE Transactions on Signal Processing.

[12]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[13]  Thomas A. Severini.  Elements of Distribution Theory , 2005 .

[14]  A. Yeredor.  MUSIC using Off-Origin Hessians of the Second Characteristic Function , 2006, Fourth IEEE Workshop on Sensor Array and Multichannel Processing, 2006..

[15]  Kazuyuki Aihara,et al.  Detecting Hidden Synchronization of Chaotic Dynamical Systems: A Kernel-based Approach , 2005 .

[16]  Hongya Ge,et al.  Direction-of-arrival estimation using distributed arrays: A canonical coordinates perspective with limited array size and sample support , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[17]  Touradj Ebrahimi,et al.  Audio-visual synchronization recovery in multimedia content , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[18]  Yoshihiro Yamanishi,et al.  Extraction of correlated gene clusters from multiple genomic data by generalized kernel canonical correlation analysis , 2003, ISMB.

[19]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[20]  K. Pearson.  Mathematical Contributions to the Theory of Evolution. III. Regression, Heredity, and Panmixia , 1896 .

[21]  Arie Yeredor,et al.  Yule-Walker Equations Applied to Hessians of the Characteristic Function for Improved AR Estimation , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[22]  Arie Yeredor,et al.  Substituting the cumulants in the super-exponential blind equalization algorithm , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[23]  K. Athreya,et al.  Measure Theory and Probability Theory , 2006 .

[24]  Gaoming Huang,et al.  Canonical Correlation Analysis Using for DOA Estimation of Multiple Audio Sources , 2005, IWANN.

[25]  M.R. Azimi-Sadjadi,et al.  Undersea Target Classification Using Canonical Correlation Analysis , 2007, IEEE Journal of Oceanic Engineering.

[26]  W. R. Buckland,et al.  A dictionary of statistical terms , 1958 .

[27]  Alon Slapak,et al.  "Weighting for more": Enhancing characteristic-function based ICA with asymptotically optimal weighting , 2011, Signal Process..

[28]  Xiangrong Yin,et al.  Canonical correlation analysis based on information theory , 2004 .

[29]  David R. Hardoon,et al.  KCCA for fMRI Analysis , 2004 .

[30]  Tunc Geveci,et al.  Advanced Calculus , 2014, Nature.

[31]  Horst Bischof,et al.  Nonlinear Feature Extraction Using Generalized Canonical Correlation Analysis , 2001, ICANN.

[32]  A. Yeredor.  Blind channel estimation using first and second derivatives of the characteristic function , 2002, IEEE Signal Processing Letters.

[33]  Arie Yeredor,et al.  Blind source separation via the second characteristic function , 2000, Signal Process..

[34]  Vince D. Calhoun,et al.  Canonical Correlation Analysis for Data Fusion and Group Inferences , 2010, IEEE Signal Processing Magazine.

[35]  Anja Vogler,et al.  An Introduction to Multivariate Statistical Analysis , 2004 .

[36]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[37]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[38]  Arie Yeredor,et al.  Blind MIMO identification using the second characteristic function , 2005 .

[39]  A. Yeredor.  Blind separation of Gaussian sources via second-order statistics with asymptotically optimal weighting , 2000, IEEE Signal Processing Letters.

[40]  Josef Kittler,et al.  Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.