Manifold-valued Dirichlet Processes

Statistical models for manifold-valued data permit capturing the intrinsic nature of the curved spaces in which the data lie and have been a topic of research for several decades. Typically, these formulations use geodesic curves and distances defined locally for most cases - this makes it hard to design parametric models globally on smooth manifolds. Thus, most (manifold specific) parametric models available today assume that the data lie in a small neighborhood on the manifold. To address this 'locality' problem, we propose a novel nonparametric model which unifies multivariate general linear models (MGLMs) using multiple tangent spaces. Our framework generalizes existing work on (both Euclidean and non-Euclidean) general linear models providing a recipe to globally extend the locally-defined parametric models (using a mixture of local models). By grouping observations into sub-populations at multiple tangent spaces, our method provides insights into the hidden structure (geodesic relationships) in the data. This yields a framework to group observations and discover geodesic relationships between covariates X and manifold-valued responses Y, which we call Dirichlet process mixtures of multivariate general linear models (DP-MGLM) on Riemannian manifolds. Finally, we present proof of concept experiments to validate our model.

[1]  Anoop Cherian,et al.  Riemannian Sparse Coding for Positive Definite Matrices , 2014, ECCV.

[2]  S. Sra,et al.  Geometric optimisation on positive definite matrices with application to elliptically contoured distributions , 2013, NIPS 2013.

[3]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[4]  P. Thomas Fletcher,et al.  Geodesic Regression and the Theory of Least Squares on Riemannian Manifolds , 2012, International Journal of Computer Vision.

[5]  Baba C. Vemuri,et al.  Statistical Analysis of Tensor Fields , 2010, MICCAI.

[6]  Vikas Singh,et al.  Canonical Correlation Analysis on Riemannian Manifolds and Its Applications , 2014, ECCV.

[7]  A. Gelfand,et al.  Dirichlet Process Mixed Generalized Linear Models , 1997 .

[8]  Warren B. Powell,et al.  Dirichlet Process Mixtures of Generalized Linear Models , 2009, J. Mach. Learn. Res..

[9]  John W. Fisher,et al.  A Dirichlet Process Mixture Model for Spherical Data , 2015, AISTATS.

[10]  Moo K. Chung,et al.  Multivariate General Linear Models (MGLM) on Riemannian Manifolds with Applications to Statistical Analysis of Diffusion Weighted Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Maher Moakher,et al.  A Differential Geometric Approach to the Geometric Mean of Symmetric Positive-Definite Matrices , 2005, SIAM J. Matrix Anal. Appl..

[12]  P. Thomas Fletcher,et al.  Principal geodesic analysis for the study of nonlinear statistics of shape , 2004, IEEE Transactions on Medical Imaging.

[13]  A. Munk,et al.  INTRINSIC SHAPE ANALYSIS: GEODESIC PCA FOR RIEMANNIAN MANIFOLDS MODULO ISOMETRIC LIE GROUP ACTIONS , 2007 .

[14]  Anuj Srivastava,et al.  Riemannian Analysis of Probability Density Functions with Applications in Vision , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  René Vidal,et al.  Sparse Riemannian manifold clustering for HARDI segmentation , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[16]  Ning Chen,et al.  Infinite SVM: a Dirichlet Process Mixture of Large-margin Kernel Machines , 2011, ICML.

[17]  Mario Medvedovic,et al.  Bayesian infinite mixture model based clustering of gene expression profiles , 2002, Bioinform..

[18]  A. Munk,et al.  Intrinsic shape analysis: Geodesic principal component analysis for Riemannian manifolds modulo Lie group actions. Discussion paper with rejoinder. , 2010 .

[19]  Anoop Cherian,et al.  Generalized Dictionary Learning for Symmetric Positive Definite Matrices with Application to Nearest Neighbor Retrieval , 2011, ECML/PKDD.

[20]  Rachid Deriche,et al.  Statistics on the Manifold of Multivariate Normal Distributions: Theory and Application to Diffusion Tensor MRI Processing , 2006, Journal of Mathematical Imaging and Vision.

[21]  Denise C. Park,et al.  A lifespan database of adult facial stimuli , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[22]  Michael I. Jordan,et al.  Matrix-Variate Dirichlet Process Priors with Applications , 2014 .

[23]  M. Escobar,et al.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[24]  J. Lafferty,et al.  Riemannian Geometry and Statistical Machine Learning , 2015 .

[25]  I. Holopainen Riemannian Geometry , 1927, Nature.

[26]  Stefan Sommer,et al.  Horizontal Dimensionality Reduction and Iterated Frame Bundle Development , 2013, GSI.

[27]  Vassilios Morellas,et al.  Dirichlet process mixture models on symmetric positive definite matrices for appearance clustering in video surveillance applications , 2011, CVPR 2011.

[28]  Xin Li,et al.  Facial Expression Recognition Influenced by Human Aging , 2013, IEEE Transactions on Affective Computing.

[29]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[30]  Baba C. Vemuri,et al.  On A Nonlinear Generalization of Sparse Coding and Dictionary Learning , 2013, ICML.

[31]  A. Kennedy,et al.  Hybrid Monte Carlo , 1987 .

[32]  Fatih Murat Porikli,et al.  Covariance Tracking using Model Update Based on Lie Algebra , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[33]  Haibin Ling,et al.  Age regression from faces using random forests , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[34]  R. Bhatia Positive Definite Matrices , 2007 .

[35]  Xavier Pennec,et al.  Intrinsic Statistics on Riemannian Manifolds: Basic Tools for Geometric Measurements , 2006, Journal of Mathematical Imaging and Vision.

[36]  Hongtu Zhu,et al.  Intrinsic Regression Models for Positive-Definite Matrices With Applications to Diffusion Tensor Imaging , 2009, Journal of the American Statistical Association.

[37]  L. Carin,et al.  The Matrix Stick-Breaking Process , 2008 .

[38]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[39]  Stefan Sommer,et al.  Optimization over geodesics for exact principal geodesic analysis , 2010, Adv. Comput. Math..

[40]  H. Ishwaran Applications of Hybrid Monte Carlo to Bayesian Generalized Linear Models: Quasicomplete Separation and Neural Networks , 1999 .

[41]  Christophe Lenglet,et al.  A nonparametric Riemannian framework for processing high angular resolution diffusion images (HARDI) , 2009, CVPR.

[42]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[43]  Radford M. Neal MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[44]  B. Afsari Riemannian Lp center of mass: existence, uniqueness, and convexity , 2011 .

[45]  R. Vidal,et al.  A nonparametric Riemannian framework for processing high angular resolution diffusion images (HARDI) , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Baba C. Vemuri,et al.  A Novel Dynamic System in the Space of SPD Matrices with Applications to Appearance Tracking , 2013, SIAM J. Imaging Sci..

[47]  Baba C. Vemuri,et al.  Recursive Karcher Expectation Estimators And Geometric Law of Large Numbers , 2013, AISTATS.

[48]  Babak Shahbaba,et al.  Nonlinear Models Using Dirichlet Process Mixtures , 2007, J. Mach. Learn. Res..