A Hessian-Free Optimization-Based Approach to Latent-Factor-Based QoS Predictors with High Accuracy

Latent-factor-based Quality-of-Service predictors can achieve high prediction accuracy and good scalability. However, most of them are based on first-order models that cannot well deal with their target problem that is inherently non-convex. Since second-order approaches have proven to be effective to such problems, this work proposes to implement a second-order predictor with an aim to achieve the high accuracy unlikely obtained by any existing methods. To do so, this work adopts the principle of Hessian-free optimization and successfully avoids the usage of a Hessian matrix by employing the efficiently obtainable product between its Gauss-Newton approximation and an arbitrary vector. Experimental results on two industrial QoS datasets indicate that the newly proposed predictor is highly accurate with fine computational efficiency.

[1]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[2]  Jorge Nocedal,et al.  On the Use of Stochastic Hessian Information in Optimization Methods for Machine Learning , 2011, SIAM J. Optim..

[3]  Barak A. Pearlmutter Fast Exact Multiplication by the Hessian , 1994, Neural Computation.

[4]  Yunni Xia,et al.  Applying the learning rate adaptation to the matrix factorization based collaborative filtering , 2013, Knowl. Based Syst..

[5]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[6]  Qingsheng Zhu,et al.  A parallel matrix factorization based recommender by alternating stochastic gradient decent , 2012, Eng. Appl. Artif. Intell..

[7]  Zibin Zheng,et al.  WSPred: A Time-Aware Personalized QoS Prediction Framework for Web Services , 2011, 2011 IEEE 22nd International Symposium on Software Reliability Engineering.

[8]  Ionel M. Navon,et al.  An Analysis of a Hybrid Optimization Method for Variational Data Assimilation , 2003 .

[9]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[10]  James Martens,et al.  Deep learning via Hessian-free optimization , 2010, ICML.

[11]  Yuki Suga,et al.  Multimodal integration learning of robot behavior using deep neural networks , 2014, Robotics Auton. Syst..

[12]  Qingsheng Zhu,et al.  Incremental Collaborative Filtering recommender based on Regularized Matrix Factorization , 2012, Knowl. Based Syst..

[13]  Dana H. Ballard,et al.  Unified losses for multi-modal pose coding and regression , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[14]  Zibin Zheng,et al.  Distributed QoS Evaluation for Real-World Web Services , 2010, 2010 IEEE International Conference on Web Services.

[15]  Jorge Nocedal,et al.  Enriched Methods for Large-Scale Unconstrained Optimization , 2002, Comput. Optim. Appl..

[16]  Zibin Zheng,et al.  QoS-Aware Web Service Recommendation by Collaborative Filtering , 2011, IEEE Transactions on Services Computing.

[17]  Zibin Zheng,et al.  Exploring Latent Features for Memory-Based QoS Prediction in Cloud Computing , 2011, 2011 IEEE 30th International Symposium on Reliable Distributed Systems.

[18]  Anne H. H. Ngu,et al.  QoS-aware middleware for Web services composition , 2004, IEEE Transactions on Software Engineering.

[19]  Hareton K. N. Leung,et al.  A Highly Efficient Approach to Protein Interactome Mapping Based on Collaborative Filtering Framework , 2015, Scientific Reports.

[20]  Ionel M. Navon,et al.  A reduced‐order approach to four‐dimensional variational data assimilation using proper orthogonal decomposition , 2007 .

[21]  Ilya Sutskever,et al.  Training Deep and Recurrent Networks with Hessian-Free Optimization , 2012, Neural Networks: Tricks of the Trade.

[22]  Jie Cao,et al.  Hybrid Collaborative Filtering algorithm for bidirectional Web service recommendation , 2012, Knowledge and Information Systems.

[23]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[24]  Zibin Zheng,et al.  WSRec: A Collaborative Filtering Based Web Service Recommender System , 2009, 2009 IEEE International Conference on Web Services.

[25]  Jorge Nocedal,et al.  Automatic Preconditioning by Limited Memory Quasi-Newton Updating , 1999, SIAM J. Optim..

[26]  Guy Shani,et al.  Evaluating Recommendation Systems , 2011, Recommender Systems Handbook.

[27]  Yehuda Koren,et al.  Advances in Collaborative Filtering , 2011, Recommender Systems Handbook.

[28]  Zibin Zheng,et al.  Collaborative Web Service QoS Prediction via Neighborhood Integrated Matrix Factorization , 2013, IEEE Transactions on Services Computing.

[29]  Bracha Shapira,et al.  Recommender Systems Handbook , 2015, Springer US.

[30]  Anne H. H. Ngu,et al.  QoS computation and policing in dynamic web service selection , 2004, WWW Alt. '04.

[31]  Quoc V. Le,et al.  On optimization methods for deep learning , 2011, ICML.

[32]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[33]  Nicol N. Schraudolph Fast Curvature Matrix-Vector Products , 2001, ICANN.

[34]  MengChu Zhou,et al.  Predicting web service QoS via matrix-factorization-based collaborative filtering under non-negativity constraint , 2014, 2014 23rd Wireless and Optical Communication Conference (WOCC).

[35]  Andrew W. Fitzgibbon,et al.  Damped Newton algorithms for matrix factorization with missing data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[36]  Pei Chen,et al.  Hessian Matrix vs. Gauss-Newton Hessian Matrix , 2011, SIAM J. Numer. Anal..

[37]  Junfeng Zhao,et al.  Personalized QoS Prediction forWeb Services via Collaborative Filtering , 2007, IEEE International Conference on Web Services (ICWS 2007).

[38]  MengChu Zhou,et al.  An Efficient Non-Negative Matrix-Factorization-Based Approach to Collaborative Filtering for Recommender Systems , 2014, IEEE Transactions on Industrial Informatics.

[39]  Zhaohui Wu,et al.  Collaborative Web Service QoS Prediction with Location-Based Regularization , 2012, 2012 IEEE 19th International Conference on Web Services.