A Non-parametric Conditional Factor Regression Model for Multi-Dimensional Input and Response

In this paper, we propose a non-parametric conditional factor regression (NCFR) model for domains with multi-dimensional input and response. NCFR enhances linear regression in two ways: a) introducing lowdimensional latent factors leading to dimensionality reduction and b) integrating the Indian Buet Process as prior for the latent layer to dynamically derive an optimal number of sparse factors. Thanks to IBP’s enhancements to the latent factors, NCFR can signicantly avoid over-tting even in the case of a very small sample size compared to the dimensionality. Experimental results on three diverse datasets comparing NCRF to a few baseline alternatives give evidence of its robust learning, remarkable predictive performance, good mixing and computational efciency.

[1]  J. Q. Smith,et al.  1. Bayesian Statistics 4 , 1993 .

[2]  Matthew West,et al.  Bayesian factor regression models in the''large p , 2003 .

[3]  Yazhen Wang,et al.  VAST VOLATILITY MATRIX ESTIMATION FOR HIGH-FREQUENCY FINANCIAL DATA , 2010, 1002.4754.

[4]  Zoubin Ghahramani,et al.  Nonparametric Bayesian Sparse Factor Models with application to Gene Expression modelling , 2010, The Annals of Applied Statistics.

[5]  M. West,et al.  High-Dimensional Sparse Factor Modeling: Applications in Gene Expression Genomics , 2008, Journal of the American Statistical Association.

[6]  Brian Neelon,et al.  Bayesian Latent Factor Regression for Functional and Longitudinal Data , 2012, Biometrics.

[7]  Simon Osindero,et al.  An Alternative Infinite Mixture Of Gaussian Process Experts , 2005, NIPS.

[8]  Hal Daumé,et al.  Multi-Label Prediction via Sparse Infinite CCA , 2009, NIPS.

[9]  Zoubin Ghahramani,et al.  Infinite Sparse Factor Analysis and Infinite Independent Components Analysis , 2007, ICA.

[10]  D. Dunson,et al.  Sparse Bayesian infinite factor models. , 2011, Biometrika.

[11]  Arnaud Doucet,et al.  Sparse Bayesian nonparametric regression , 2008, ICML '08.

[12]  Qiang Wu,et al.  Support vector regression for multi-view gait recognition based on local motion feature selection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[14]  Yee Whye Teh,et al.  Semiparametric latent factor models , 2005, AISTATS.

[15]  Thomas L. Griffiths,et al.  Infinite latent feature models and the Indian buffet process , 2005, NIPS.

[16]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[17]  Thomas L. Griffiths,et al.  The Indian Buffet Process: An Introduction and Review , 2011, J. Mach. Learn. Res..

[18]  Cristian Sminchisescu,et al.  Supervised Spectral Latent Variable Models , 2009, AISTATS.

[19]  Moritz Tenorth,et al.  The TUM Kitchen Data Set of everyday manipulation activities for motion tracking and action recognition , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[20]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .