Multiple-Output Regression with High-Order Structure Information

In this paper, we propose a new method to learn the regression coefficient matrix for multiple-output regression, which is inspired by multi-task learning. We attempt to incorporate high-order structure information among the regression coefficients into the estimated process of regression coefficient matrix, which is of great importance for multiple-output regression. Meanwhile, we also intend to describe the output structure with noise covariance matrix to assist in learning model parameters. Taking account of the real-world data often corrupted by noise, we place a constraint of minimizing norm on regression coefficient matrix to make it robust to noise. The experiments are conducted on three public available datasets, and the experimental results demonstrate the power of the proposed method against the state-of-the-art methods.

[1]  A. Rukhin.  Matrix Variate Distributions , 1999, The Multivariate Normal Distribution.

[2]  Ming-Hsuan Yang,et al.  Discriminative Generative Contour Detection , 2013, BMVC.

[3]  ChengXiang Zhai,et al.  Robust Unsupervised Feature Selection , 2013, IJCAI.

[4]  Dit-Yan Yeung,et al.  Multi-task warped Gaussian process for personalized age estimation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Guodong Guo,et al.  Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression , 2011, CVPR 2011.

[6]  Lin Li,et al.  Multi-output least-squares support vector regression machines , 2013, Pattern Recognit. Lett..

[7]  Jan Peters,et al.  Model Learning with Local Gaussian Process Regression , 2009, Adv. Robotics.

[8]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[9]  Kyung-Ah Sohn,et al.  Joint Estimation of Structured Sparsity and Output Structure in Multiple-Output Regression via Inverse-Covariance Regularization , 2012, AISTATS.

[10]  Dit-Yan Yeung,et al.  A Convex Formulation for Learning Task Relationships in Multi-Task Learning , 2010, UAI.

[11]  Eric P. Xing,et al.  A multivariate regression approach to association analysis of a quantitative trait network , 2008, Bioinform..

[12]  I-Cheng Yeh,et al.  Modeling slump flow of concrete using second-order regressions and artificial neural networks , 2007 .

[13]  Hal Daumé,et al.  Simultaneously Leveraging Output and Task Structures for Multiple-Output Regression , 2012, NIPS.

[14]  Jeff G. Schneider,et al.  Learning Multiple Tasks with a Sparse Matrix-Normal Penalty , 2010, NIPS.

[15]  Adam J Rothman,et al.  Sparse Multivariate Regression With Covariance Estimation , 2010, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[16]  J. Friedman,et al.  Predicting Multivariate Responses in Multiple Linear Regression , 1997 .

[17]  Ming-Hsuan Yang,et al.  Contour detection via random forest , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[18]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[19]  Dit-Yan Yeung,et al.  Learning High-Order Task Relationships in Multi-Task Learning , 2013, IJCAI.

[20]  Yufeng Liu,et al.  Simultaneous multiple response regression and inverse covariance matrix estimation via penalized Gaussian maximum likelihood , 2012, J. Multivar. Anal..